Custom Converter Use Case: Conversion of Large XML file to CSV
In this article, we describe one of the use-cases of our client who wanted to convert Large XML files into CSV format and what solution was provided - the Custom Converter.Problem
Our client wanted to convert large (up to 50 GB) XML files into CSV format.
Additional requirements were:
- only specific fields (XML Nodes) should be extracted in the output
- some of the columns (in CSV) should be renamed according to the provided mapping
Solution: Custom Converter
After analyzing the requirements we suggested developing a Custom Converter, which is based on the regular XML to CSV Converter but could perform the required data processing tasks:
- Data Filtering
- Fields Mapping
Data Filtering
The first data processing task was to pass to the output only the specific fields (or XML nodes).
This feature performing a data filtering task.
During the processing of the XML file, based on the XML file structure and on the provided list of fields to filter, Custom Converter performing a data filtering task skipping unwanted data on the early stage.
Fields Mapping
Another task was to re-map fields (XML Nodes) to have different names from the provided list.
This feature performing a fields mapping task.
During the analyzing the XML file structure, Custom Converter assigning different names to the result columns based on the provided list.
Summary
As a result, our solution developed as Custom Converter performing required tasks of filtering data and field mapping of the large XML file when converting it into CSV format.
This helped not only to reduce the result CSV file size but also significantly reduced the resource usage by the server for the conversion process which uses less time to process large XML files.
Get a Free Quote
Would you like to process your files more efficiently?
Get a Free Quote right now using the form below! We usually reply within the next few hours.
Share this article with your friends or colleagues!
Custom Converter Use Case: Conversion of Large XML file to CSV
In this article, we describe one of the use-cases of our client who wanted to convert Large XML files into CSV format and what solution was provided - the Custom Converter.Problem
Our client wanted to convert large (up to 50 GB) XML files into CSV format.
Additional requirements were:
- only specific fields (XML Nodes) should be extracted in the output
- some of the columns (in CSV) should be renamed according to the provided mapping
Solution: Custom Converter
After analyzing the requirements we suggested developing a Custom Converter, which is based on the regular XML to CSV Converter but could perform the required data processing tasks:
- Data Filtering
- Fields Mapping
Data Filtering
The first data processing task was to pass to the output only the specific fields (or XML nodes).
This feature performing a data filtering task.
During the processing of the XML file, based on the XML file structure and on the provided list of fields to filter, Custom Converter performing a data filtering task skipping unwanted data on the early stage.
Fields Mapping
Another task was to re-map fields (XML Nodes) to have different names from the provided list.
This feature performing a fields mapping task.
During the analyzing the XML file structure, Custom Converter assigning different names to the result columns based on the provided list.
Summary
As a result, our solution developed as Custom Converter performing required tasks of filtering data and field mapping of the large XML file when converting it into CSV format.
This helped not only to reduce the result CSV file size but also significantly reduced the resource usage by the server for the conversion process which uses less time to process large XML files.
Get a Free Quote
Would you like to process your files more efficiently?
Get a Free Quote right now using the form below! We usually reply within the next few hours.
Share this article with your friends or colleagues!