Query Optimisation and Near-Data Processing on Reconfigurable SoCs for Big Data Analysis

Query Optimisation and Near-Data Processing on Reconfigurable SoCs for Big Data Analysis

(Third Party Funds Group – Sub project)

Overall project: DFG Priority Programme (SPP) 2037 - Scalable Data Management for Future Hardware
Project leader: Stefan Wildermann, Jürgen Teich, Klaus Meyer-Wegener
Project members: Tobias Hahn, Maximilian Langohr, Andreas Becher, Lekshmi Beena Gopalakrishnan Nair
Start date: 28/08/2017
End date: 01/03/2023
Acronym: ReProVide
Funding source: DFG / Schwerpunktprogramm (SPP)
URL: https://www.dfg-spp2037.de/me943-9/

Abstract

This project is funded by the German Research Foundation (DFG) within the Priority Program SPP 2037 "Scalable Data Management for Future Hardware".

The goal of this project is to provide novel hardware and optimisation techniques for scalable, high-performance processing of Big Data. We particularly target huge data sets with flexible schemata (row-oriented, column-oriented, document-oriented, irregular, and/or non-indexed) as well as data streams as found in click-stream analysis, enterprise sources like emails, software logs and discussion-forum archives, as well as produced by sensors in IoT and Industrie 4.0. In this realm, the project investigates the potential of hardware-reconfigurable, FPGA-based SoCs for near-data processing where computations are pushed towards such heterogeneous data sources. Based on FPGA technology and in particular on their dynamic reconfiguration, we propose a generic architecture called ReProVide for low-cost processing of database queries.

The concepts are intended to enable the integration of FPGA-based acceleration support into available SQL, NoSQL, and in-memory database management systems (DBMSs) as well as stream-processing frameworks. Our intention is to attach volatile and non-volatile storage directly to ReProVide nodes, which will not only contain cleansed and integrated data sets, but can also be used for temporarily or persistently storing uncleaned data from new data sources and data streams.

Our FPGA-based SoC is psuhed forward by the Chair for Computer Science 12. It

• makes use of hardware reconfiguration to adapt datapaths and accelerators for being able to process different OLAP and data-mining operators on data from such heterogeneous data sources,

• provides management techniques to generate local meta-data, indexes, and statistics of such data sources for optimised data processing, as well as

• offers schema-on-read capabilities for the DBMS accessing the SoC.
  

While the support of irregular data (e.g., graph processing) is not in the main focus of our research, we provide a design methodology that is generic and extensible by user-defined functions and data schemata.

Integration of such architectures that come with their own local optimiser into DBMS requires novel global query optimisation techniques based on concepts known from multi-databases research. This is the task of the Chair for Computer Science 6. While the local optimiser builds statistics of its local data, the global optimiser has to access such data and information of this near-data processor. Global query optimisation decides based on these data which operations are worthwhile to assign to ReProVide SoCs, and which are not. It is vital that the optimiser has enough knowledge to engage ReProVide in query processing whenever there is a benefit. This requires functional knowledge (which data and which operators can be offered) as well as non-functional knowledge (e.g., cost estimates). In this project, we provide an extensible interface over which not only the global optimiser can hand over the QEP to be processed on the ReProVide system and the ReProVide can transmit the query result. But it will also enable to bidirectionally exchange hints to improve their respective optimisation.

Publications

• Hahn T., Becher A., Wildermann S., Teich J.:
  Raw Filtering of JSON data on FPGAs
  Design, Automation and Test in Europe Conference (DATE) (Antwerpen, 14/03/2022 - 23/03/2022)
  In: Proceedings of the 2022 Conference & Exhibition on Design, Automation & Test in Europe 2022
  DOI: 10.23919/DATE54114.2022.9774696
  BibTeX: Download
• Hahn T., Wildermann S., Teich J.:
  Auto-Tuning of Raw Filters for FPGAs
  International Conference on Field-Programmable Logic and Applications (FPL) (Belfast, United Kingdom, 29/08/2022 - 02/09/2022)
  In: IEEE Proceedings of the 32nd International Conference on Field-Programmable Logic and Applications 2022
  DOI: 10.1109/FPL57034.2022.00036
  BibTeX: Download
• Beena Gopalakrishnan Nair L., Becher A., Meyer-Wegener K., Wildermann S., Teich J.:
  SQL Query Processing Using an Integrated FPGA-based Near-Data Accelerator in ReProVide
  23rd International Conference on Extending Database Technology (Copenhagen, 30/03/2020 - 02/04/2020)
  In: Proceedings of EDBT 2020
  BibTeX: Download
• Beena Gopalakrishnan Nair L., Becher A., Meyer-Wegener K.:
  The ReProVide Query-Sequence Optimization in a Hardware-Accelerated DBMS
  16th International Workshop on Data Management on New Hardware Held with ACM SIGMOD/PODS 2020 (Portland, Oregon USA, 15/06/2020 - 15/06/2020)
  In: DaMoN '20: Proceedings of the 16th International Workshop on Data Management on New Hardware 2020
  DOI: 10.1145/3399666.3399926
  BibTeX: Download
• Becher A., Teich J.:
  In situ Statistics Generation within partially reconfigurable Hardware Accelerators for Query Processing
  15th International Workshop on Data Management on New Hardware (DaMoN) Held with ACM SIGMOD/PODS 2019 (Amsterdam, 01/07/2019 - 01/07/2019)
  DOI: 10.1145/3329785.3329936
  BibTeX: Download