Hitachi today announced the development of a database management system optimized for the high-speed embedded memory in the hardware (FPGA) and technology for high performance parallel data processing in FPGAs. Using these technologies, speed of data analytics was successfully increased by up to a maximum of 100 times compared with not using these technologies. In recent years, self-service analytics that allow employees in the field to easily conduct big data analytics, usually conducted by experts such as data scientists, is gaining attention. One example might be that of a financial advisor, listening to a customer requirements, entering the information into the analytics system on the spot, and being able to suggest a financial product which matches the customer’s needs.
As can be imagined from this example, the data analytics system for self-service data analytics needs to produce results quickly, and thus must have high processing capabilities to execute data read and data analysis processes. By using flash storage instead of a hard disk drive to store data, the data read performance is increased by up to 10 to 100 times. Data analysis performance, however, has been unable to keep up with data read performance, thus creating a bottleneck in the analytics.To overcome this issue, Hitachi developed a database management system optimized for the high-speed embedded memory in the hardware (FPGA) and technology to conduct high speed parallel data processing in the FPGAs, and succeeded in increasing data analytics speed by up to a maximum of 100 times. A real-time data analytics system prototype was then built by combining these two technologies with Pentaho Business Analytics for visualization of results, and flash storage for data store.
