support
Middleware to Alleviate Database Pressure

esProc is the middleware designed specifically to undertake the computing jobs between database and application. For the application, esProc provides the easy-to-use JDBC interface; For the database, esProc is powerful in parallel computing. By implementing the computing outside the database and in external storage, esProc alleviates the computing, storage and concurrency pressure on the database; effectively cuts down the cost of database software and hardware; and greatly optimizes database management.

 

Alleviating computing pressure. Too much computing will occupy a lot of CPU utilization and memory resources of the database server. Such cases include the data-intensive computing and computation-intensive tasks involving only a small amount of data. Consequently, long response time will deteriorate user experience and the overloading will shorten the life of the server. esProc has a complete and well-defined computing system, making it fully capable of sharing the workload for the database, and easily undertake various computing tasks, difficulty or easy, from the application. In addition, esProc supports parallel computing across multiple nodes, enabling it to equally distribute the task involving massive data or computations to multiple average servers or inexpensive PCs. In this way, users can achieve an overall outstanding performance much more superior to database servers while keeping a far lower provision of node hardware than that of the database servers.

Alleviating storage pressure. For the sake of safety, performance and usability, all kinds of application systems will not manipulate the core business data directly. Instead, the core business data will be processed into the form of temporary table/intermediate table for the direct use by application. Such redundant data and the associated SP would occupy a great many of storage spaces in the database, resulting in great storage pressure on database. esProc features the mechanism of external memory computing, which stores the intermediate database data into files on inexpensive HDDs and thus remove the need for database storage spaces. Through esProc JDBC, applications can access to the data stored in external storages and perform the standard structured data operations like query, compute, and modify. esProc also supports the external storage file retrieval and write-back to HDFS, which has a much greater storage space than that of any high capacity database.

Alleviating concurrency pressure. The enterprise level report applications or Web applications generally involve a great number of concurrent accesses and ultimately incur great pressure on database, which would reject the computing requests from users because all connection quotas have been used up. On the one hand, it is an effective and expensive means to solve the problem by upgrading the database hardware and buying more user licenses. On the other hand, limiting the number of concurrent connections and reducing the response will cut down the database cost while compromising the user experience and productivity. esProc solves the dilemma that the database cost can be lowered while alleviating the pressure of concurrency on database by supporting the parallel computing. The compute-intensive task can now be equally divided and allocated to multiple average PCs. Each node only needs to undertake a small amount of computing jobs.

Data administration tree. A database is a flat directory structure where all data tables are placed together linearly. Such situation is also true with SP. What’s worse, the intermediate table and its corresponding SP can only be stored separately in different directories. The flat directory structure is the root of disorder in data management/permission and application management. The public and private SPs become entangled and the intermediate tables for the sales or the production application get messed up. By comparison, esProc features the trouble-free external memory computing that manages source files and script files with an organized file system in a directory tree, where data can be placed in different directories according to the category and hierarchy and various sub-directories will be arranged for various modules, names of directories clearly indicate and differentiate the data from the new and the legacy systems, the script file and its source data can be stored in the same place, and the import, export, backup and recovery can be done with several simple actions of copy & paste. With esProc, data management becomes simple and clear.

Cutting database cost. With esProc users can store the core data in the database, with relatively low cost in management and hardware and fewer user licenses for maintaining and calculating important data and security management. The intermediate tables and script files derived from the core data are all stored outside the database. By doing so, the high-concurrency, big data parallel computing can be implemented on the inexpensive PCs. The data expansion can be solved easily with the HDFS and inexpensive HDDs. By reasonably leverage resources, the workload pressure on database can be significantly reduced, and database cost kept under control, management problems solved effectively, and various database applications handled with ease. These applications include the real-time high performance application, non-real-time big data application, desktop BI, report application, and ETL.

esProc enables users to write codes following natural thinking straightforwardly, not having to change their train of thoughts in converting the complex business logics into practical codes. To complete the same computing, esProc is several timesmore neat and concise than Java and SQL. ...
esProc is specialized incomputing from various different data sources, and the computing result can be written back to multiple or single data source. esProc ont only supports the rational database, MongoDB, Cassandra,and other NoSQL database, but also provides a great many functions to handle the structureddata and the semi-structureddata computing....
esProc supports the multi-thread computing on single node and parallel computing on multiple nodes without center. The big data can be divided into several small data blocks, then used in multiple node machine for parallel computing, and lastly merging andcomputing....
esProc optimizes the syntax for the structureddata, supports the in-memory computing and ordered set, and enables programmers to select the optimized path based on the characteristics of data and algorithms. The actual measured data indicates that the performance of esProc is close or even higher than that of database for the standalone machine. ...
esProc is the pure Java product, with the outstanding integration capablity by nature. esProc can be divided into four sections: JDBC, Command line, Server, and IDE. The JDBC is the specific calling interface for applications. The invoking method of esProc JDBC is similar to the stored procedure of normal database, i.e....
esProc is a data computing language with the powerful TSeq and cursor data object. It is especially optimized for the computing over (semi) structureddata, and capable of handling various complex computing problems easily. Because it is designed to serve the sole purpose of computing, esProc is simply structured and easy to grasp. ...
esProc is designed for application programmers to handle computing strongly related to the typical and complex business facts. Without any inheriting, overload, and other complex concepts, it is a development language neither object-oriented, nor fit for developing the infrastructure or the whole tool software. ...
Reorganization of the report data source: esProc is especially fit for the computing involving the complex business algorithm of data sources or the cross-database computing involving multiple data sources....
esProc is the professiona