Search Engine on Master Database
First, let’s briefly understand the data retrieval in the era of big data.
In today’s big data era, the amount of information is exploding, and the data is growing rapidly. How to realize the rapid query and retrieval of information on the amount of data that Storage unit is T or P has become one of the core propositions in big data technology.
Under the big data technology system, information management departments generally choose to use the mainstream open-source component ELK as the query and retrieval channel for unstructured data. ELK is composed of Elasticsearch、Logstash and Kibana components.
Elasticsearch, or ES for short, is an open-source distributed search engine. Its features include distribution, zero configuration, automatic discovery, automatic index fragmentation, index copy mechanism, restful style interface, multiple data sources, automatic search load, etc.
Logstash is a completely open-source tool that can collect, analyze, and store your logs for future use.
Kibana is an open source and free tool. It can provide Logstash and Elasticsearch with a friendly Web interface for log analysis, which can help you summarize, analyze and search important data logs.
After a brief understanding of the data retrieval under the big data system, we return to the content involved in the title of the article. For traditional databases, generally, we will use SQL statements to query, count and aggregate the information in the database. This technology has been teaching and preaching in the basic course database principle of the University. For IT people, Using SQL statements to interact with the database becomes natural and natural.
However, let’s imagine, in many cases, when we look for a data, what will we do?
- I need to know which data table the data is in, and then I write SQL statements to query
- What if you forget the table name? I’m going to find the document and check the database structure
- What if there is no document? I will use select * from tab where TABLE_ NAME like ‘%%’ filter what you can think of first.
- What should I do if I don’t know? Find a service provider or developer to help me find it, but can’t find it? If you can’t find it, you can’t. keep looking, but this time may have passed a week.
- If I find the data I want to find, but find that the fields related to the data are all kinds of codes, I can’t understand it? How to translate the code? Well, we must go through 1, 2, 3 and 4 again.
Now, if there is a search engine for all the table names, field names, field contents and associated
table information in the database, enter the content you want to fuzzy query on the search engine at will. It may be the approximate name of a field you remember, or it may be a person’s name. The search result is all the matching data tables, data fields, field contents and associated data information related to your search keywords.
Does that sound smart?
Now we have achieved the goal. Adhering to the persistence of data technology and the concept of creating value for customers, we have combined master data management, database, and search engine to provide the simplest and most direct database query method for data administrators. We hope to make data management simpler and make the work of Data Administrators easier.
Associated Data Query
We will continue our exploration in data technology and look forward to your continued attention to our company.
Be the most professional data technology team to make the world clearer