Semi-structured data is the data which does not conforms to a data model but has some structure. It lacks a fixed or rigid schema. It is the data that does not reside in a rational database but that have some organisational properties that make it easier to analyse. With some process, we can store them in the relational database.
Characteristics of semi-structured Data:
- Data does not conforms to a data model but has some structure.
- Data can not be stored in the form of rows and columns as in Databases
- Semi-structured data contains tags and elements (Metadata) which is used to group data and describe how the data is stored
- Similar entities are grouped together and organised in a hierarchy
- Entities in the same group may or may not have the same attributes or properties
- Does not contains sufficient metadata which makes automation and management of data difficult
- Size and type of the same attributes in a group may differ
- Due to lack of a well defined structure, it can not used by computer programs easily
Sources of semi-structured Data:
- XML and other markup languages
- Binary executables
- TCP/IP packets
- Zipped files
- Integration of data from different sources
- Web pages
Advantages of Semi-structured Data:
- The data is not constrained by a fixed schema
- Flexible i.e Schema can be easily changed.
- Data is portable
- It is possible to view structured data as semi-structured data
- Its supports users who can not express their need in SQL
- It can deal easily with the heterogeneity of sources.
Disadvantages of Semi-structured data
- Lack of fixed, rigid schema make it difficult in storage of the data
- Interpreting the relationship between data is difficult as there is no separation of the schema and the data.
- Queries are less efficient as compared to structured data.
Problems faced in storing semi-structured data
- Data usually has an irregular and partial structure. Some sources have implicit structure of data, which makes it difficult to interpret the relationship between data.
- Schema and data are usually tightly coupled i.e they are not only linked together but are also dependent of each other. Same query may update both schema and data with the schema being updated frequently.
- Distinction between schema and data is very uncertain or unclear. This complicates the designing of structure of data
- Storage cost is high as compared to structured data
Possible solution for storing semi-structured data
- Data can be stored in DBMS specially designed to store semi-structured data
- XML is widely used to store and exchange semi-structured data. It allows its user to define tags and attributes to store the data in hierarchical form.
Schema and Data are not tightly coupled in XML.
- Object Exchange Model (OEM) can be used to store and exchange semi-structured data. OEM structures data in form of graph.
- RDBMS can be used to store the data by mapping the data to relational schema and then mapping it to a table
Extracting information from semi-structured Data
Semi-structured data have different structure because of heterogeneity of the sources. Sometimes they do not contain any structure at all. This makes it difficult to tag and index. So while extract information from them is tough job. Here are possible solutions –
- Graph based models (e.g OEM) can be used to index semi-structured data
- Data modelling technique in OEM allows the data to be stored in graph based model. The data in graph based model is easier to search and index.
- XML allows data to be arranged in hierarchical order which enables the data to be indexed and searched
- Use of various data mining tools
To read Differences between Structured, Semi-structured and Unstructured data refer the following article –
Attention reader! Don’t stop learning now. Get hold of all the important CS Theory concepts for SDE interviews with the CS Theory Course at a student-friendly price and become industry ready.
- Data Abstraction and Data Independence
- Types of Sources of Data in Data Mining
- Characteristics of Biological Data (Genome Data Management)
- Difference between Data Warehousing and Data Mining
- Data Preprocessing in Data Mining
- Difference between Data Warehouse and Data Mart
- Data Mining: Data Warehouse Process
- Data Mining: Data Attributes and Quality
- Data Reduction in Data Mining
- Data Transformation in Data Mining
- Difference between Data Lake and Data Warehouse
- Data Architecture Design and Data Management
- Difference between Data Privacy and Data Security
- Difference between Data Privacy and Data Protection
- Difference between Traditional data and Big data
- Data Mining
- Data Warehousing
- Data Replication in DBMS
- KDD Process in Data Mining
- Snowflake Schema in Data Warehouse Model