What is a big data database?
There is no difference between normal data and big data apart from size. Both are holding data, but compare to normal databases, Big data bases can handle Structured and unstructured data including images, videos, log data, IoT data etc., The smallest unit of measurement used for measuring data is a bit.
Below is a list of all the standard units of measurement used for data storage, from the smallest to the largest.
What units of measurement are used for data storage?
Measurement of Data Units in the Data Storage Or Big Data Storage Measuring Units
Size in Bytes
Up to gigabyte we can store the data Systems or Serves but in case of Terabytes or More then that data can not be stored in single system, and we need to super computers to store the huge data. It has become clear that the traditional relational data management systems (RDBMS) not able to handle this huge data. As result a variety of big data database options have emerged. To overcome the limitations of the RDBMS database, there are different technologies available to handle to extract values form this huge data.
Most of the Case Systems that designed for bigdata is called as NoSQL (Not only SQL), which is enough capable to handle in terms of Volume, Velocity, variability and Speed.
NoSQL databases are growing Very Rapid Speed Because of their exciting features Like;
- Schema free Architecture
- Easy Replication
- Support for Big Data
- Simple API
What are the Different types of Databases in the market to fulfill the Big data requirements?
There are different Kinds of big data databases in the market based on requirements
Basically we can bifurcate as bellow based on there nature of structure
1. Key-Values Stores
The Key-Value is well known concept in the many programming languages key-value as an associative array or data structure. Using hash table where there us unique key and a pointer to a particular item of data.
Examples: Tokyo Cabinet/Tyrant, Redis, Voldemort, Oracle Berkeley DB, Amazon SimpleDB, Riak, Oracle NoSQL Database
2. Column Family Stores
Some key benefits of columnar databases include:
• Compression. Column stores are very efficient at data compression and/or partitioning.
• Aggregation queries. Due to their structure, columnar databases perform particularly well with aggregation queries (such as SUM, COUNT, AVG, etc).
• Scalability. Columnar databases are very scalable. They are well suited to massively parallel processing (MPP), which involves having data spread across a large cluster of machines – often thousands of machines.
• Fast to load and query. Columnar stores can be loaded extremely fast. A billion row table could be loaded within a few seconds. You can start querying and analysing almost immediately.
These are just some of the benefits that make columnar databases a popular choice for organisations dealing with big data.
These were created to store and process very large amounts of data distributed over many machines. There are still keys but they point to multiple columns. The columns are arranged by column family.
A column store database is a type of database that stores data using a column-oriented model.
A column store database can also be referred to as a:
• Column database
• Column family database
• Column oriented database
• Wide column store database
• Wide column store
• Columnar database
• Columnar store
Examples: Cassandra, HBase, Bigtable, Cassandra, HBase, Vertica, Druid, Accumulo, Hypertable
3. Document Databases
These were inspired by Lotus Notes and are similar to key-value stores. The model is basically versioned documents that are collections of other key-value collections. The semi-structured documents are stored in formats like JSON. Document databases are essentially the next level of Key/value, allowing nested values associated with each key. Document databases support querying more efficiently.
NoSQL databases can store non-relational data on a super large scale, and can solve problems regular databases can’t handle: indexing the entire Internet, predicting subscriber behavior, or targeting ads on a platform as large as Facebook. But with over 150 NoSQL database types, it can be hard for a SQL professional to know where to start. In this course, Lynn Langit breaks these types down into five main categories and shows how to get your own NoSQL database up and running with easy-to-configure cloud solutions. You’ll learn how to add and query data and also examine case studies where NoSQL was used to solve real-world data storage management issues. The final chapter contains tips just for startup businesses that are considering their first NoSQL solution.
Document databases store data in the form of semi-structured documents, such as JSON or XML. Like key-value stores, these documents are free of schemas and can be accessed with a key called the unique document identifier. Unlike key-value stores, document databases have transparent values, allowing for indexing and content-based querying. Document databases are built around the idea of keeping related data in the same document when possible. This leads to fast reads, but may require some redundancy for certain data structures
Examples: CouchDB, MongoDb
4. Graph Databases
Instead of tables of rows and columns and the rigid structure of SQL, a flexible graph model is used which, again, can scale across multiple machines. NoSQL databases do not provide a high-level declarative query language like SQL to avoid overtime in processing. Rather, querying these databases is data-model specific. Many of the NoSQL platforms allow for RESTful interfaces to the data, while other offer query APIs.
Examples: Neo4J, InfoGrid, Infinite Graph
Types of Big Data Database by Data Storage Methods and list of NoSQL databases which supports the Big data Database.
List of Wide Column Stores/Column Family Databases
- Amazon SimpleDB
- Cloud Data
List of Key Value / Tuple Store databases
- Amazon DynamoDB
- Azure Table storage
- Berkeley DB
- Oracle NoSQL Database
- Tokyo Cabnit/Tyrant
- c-treeACE database
- TIBCO ActiveSpaces DB
- Symas Lightning Memory Mapped Database (LMDB)
- Light Cloud
List of Multimodel Databases
List of XML Databases:
- EMC Documentum xDB
List of Network Model Databases
List of Document Store Database
- Elastic Search
- Couchbase Server
- MarkLogic Server
- Clusterpoint Server
- Amisa Server
List of Graph Databases
- WHITE Database
List of Object Databases
- HSS Database
List of Grid & Cloud Database
- Oracle Coherence
List of Multidimensional Databases
- Intersystem cache