Data is Growing

NEW YORK TIMES
Big Data is getting a $70 billion industry and growing at a rapid rate of 15% to 20% a year. Big Data developments will be perhaps the most critical new marketplace for storage solutions providers in the coming decade. Providing a strong portfolio of complete Big Data solutions – hardware, software, and implementation services – will be a high priority to succeed.
In reality, the concept of Big Data has really kicked in with the rise of cloud-based service providers like iTunes, Netflix, Youtube, Facebook etc. which are primarily in the business of gathering, organizing, and providing access to large quantities of digital content. In addition, cloud-based service providers like Salesforce.com, Amazon web services and others that have focused on delivering IT infrastructure and applications “as a service”.
Over the past several years, these companies have undertaken massive storage build outs as they have expanded their service offerings, entered new markets, and extended their geographic reach.
In parallel to the expansion of the public cloud, many organizations have started to deploy their own private clouds for internal applications and content (archival) storage. Some of these private cloud deployments (e.g., government and research sites) are comparable in scope and complexity to public cloud environments while others are more limited.
In reality, the concept of Big Data has really kicked in with the rise of cloud-based service providers like iTunes, Netflix, Youtube, Facebook etc. which are primarily in the business of gathering, organizing, and providing access to large quantities of digital content. In addition, cloud-based service providers like Salesforce.com, Amazon web services and others that have focused on delivering IT infrastructure and applications “as a service”.
Over the past several years, these companies have undertaken massive storage build outs as they have expanded their service offerings, entered new markets, and extended their geographic reach.
In parallel to the expansion of the public cloud, many organizations have started to deploy their own private clouds for internal applications and content (archival) storage. Some of these private cloud deployments (e.g., government and research sites) are comparable in scope and complexity to public cloud environments while others are more limited.
Big Data Wave

ACCEL
Public and private cloud storage demand drives growth
The spending on storage by public cloud service providers is expected to grow at an annual rate of 24% from 2010 to 2015 while enterprise spending on storage for the private cloud will experience annual growth of near 30%.
Though the ‘Big Data’ phenomenon seems to have gained popularity only after the launch of cloud services, in reality, some aspects of this emerging trend are not new.
The data volumes have grown exponentially for several years now and, in fact, a major chunk of the ‘Big Data’ currently housed in massive data centers have nothing to do with cloud services.
The spending on storage by public cloud service providers is expected to grow at an annual rate of 24% from 2010 to 2015 while enterprise spending on storage for the private cloud will experience annual growth of near 30%.
Though the ‘Big Data’ phenomenon seems to have gained popularity only after the launch of cloud services, in reality, some aspects of this emerging trend are not new.
The data volumes have grown exponentially for several years now and, in fact, a major chunk of the ‘Big Data’ currently housed in massive data centers have nothing to do with cloud services.
Big Data Analysis
The big data analysis technology stack is divided into six layers:
- Integration frameworks
- Management frameworks
- Modeling frameworks (i.e., Avro)
- Development frameworks (i.e., Pig and Hive)
- Processing frameworks (i.e, Hadoop and Oozie)
- Data management frameworks (i.e, HDFS and HBase)
We partner with Hadoop vendors to offer the following;
- Translation of large texts from one language into another; we could break each document into paragraphs, and send each paragraph to a different node for translation.
- Apply a common process to a large number of files. (The New York Times's impressive combination of Hadoop and Amazon's EC2. http://open.blogs.nytimes.com/tag/hadoop/ )
- Travel planning. Find the optimal price and speed for travel between two points, given many different transportation options and mobile data. Each node can calculate the time, distance, and expense for a different combination of travel methods.
- Massive compilations. If you need to compile several thousand files, each of which takes several minutes, but isn't connected to other files to be compiled, you can farm the compilation out to various nodes with Hadoop.
- Price calculations. If you have thousands of items in an online product catalog, and if the prices need to change depending on your competitor's prices, you could use Hadoop to reprice your products in parallel.
- Stock history analysis, as well as automatic trading. Instead of looping over a list of stocks, and then looking up the history of each one, create a Hadoop cluster and have each node look a the history of a different stock. Use the "reduce" function to find those stocks that have gone up or down substantially, and use that data to either buy or sell. If you really had supercomputing type power at hand, you could do much more sophisticated things like arbitrage of options and futures. It depends how fast the answers come back, but if it's fast then you could beat a lot of other market players in short term opps.
- Collaborative filtering - you need to mine through reams of customer data and build data structures that can be queried later. A lot of the crunching can be parallelized.
- Data Indexing - different parts of the data corpus can first be broken up into separate marts and then be indexed.
- Data analysis, ex: SETi At Home http://setiathome.berkeley.edu
- Carrier DataBase Organization
Who is MiSONE
|
Highlights
|
Follow Us On
|
Worldwide
|
|
Copyright © 2015 MISONE.COM.TR | MİSONE HABERLEŞME TEKNOLOJİ VE YAZILIM A.Ş.
|