Hyper Open Edge Cloud

How To Import CSV Data

How To demonstrating how to import CSV files into ERP5.
  • Last Update:2016-02-11
  • Version:001
  • Language:en

How to demonstrating how to ingest CSV data into ERP5

Table of Contents

System Preparation

First install ERP5 and some modules, then install erp5_csv_style Business Template. Now create Script (Python) object with id Base_importCsvLine in portal_skins/custom folder with content: 

if object_property_dict.has_key('uid'):
  object = context.portal_catalog.getObject(object_property_dict['uid'])
else:
  object = context.newContent()
 
# activity doesn't support security rights yet...
for key in ['uid','id']:
  if object_property_dict.has_key(key):
    object_property_dict.pop(key)

object.edit(**object_property_dict)

(For now original version of Base_importCsvLine has little problem).

CSV File Preparation

Create CSV file, which is like in this example: 

"title" "corporate_name"  "vat_code"  "default_address_region"  "default_address_city"  "role"
"title" "corporate_name"  "vat_code"  "default_address_region"  "default_address_city"  "role"
"Ventis"  "Ventis s.c."  "1234567889"          "1/1"               "Tarnowskie Góry" "producer"

Note:

  • First two lines containing name of fields, which you would like to import, eg in case of Organisation (TODO: second will be ignored, isn't it?):
  • Third and next lines are with data, like
  • Every line shall be separated with enter. To have special characters in your data
    • replace " with ""
    • replace newline-character with TODO (I suggest to replace newline with some special combination of charaters, to made it post-import praseable.)

Problems when importing

The best way to analyze problems is to turn off activities. To do so replace: 

method(priority=4, activity="SQLQueue", passive_commit=1).Base_importCsvLine(attribute_value_dict)

with:

context.Base_importCsvLine(attribute_value_dict)

in Base_importCsvFile. (Of course copy it from erp_core skin folder to custom before customization).

Note: importing take some time. Long time sometimes - in our case it took longer than writing this wiki-page and drinking much of wine (working on Duron 600 with 768MB of RAM).

Special thanks to Tomasz Brzezina for support and testing data.

Related Articles