CSV Processing Configuration Options

The CSV Processing Operation in Operation pipeline supports the following configuration:

  • Quote character

  • Delimiter character

  • End of Line Symbol

In order to apply this configuration, the following steps need to be done:

  1. Dashboard → Routes → SFTP → Manage

    mceclip0.png

     

  2. Export the route

    mceclip1.png
     
  3. Open the exported route in a text editor and find the operations tag:

    mceclip2.png
     
  4. Set the following value for the operations tag:

              operations: |-
                [ {
                  "@type" : "type.googleapis.com/ProcessInputStreamPairOperationConfig"
                }, {
                  "@type" : "type.googleapis.com/ReplaceCsvColumnOperationConfig",
                  "hasHeader" : true,
                  "quoteChar" : "`",
                  "columnIndices" : [ 3 ],
                  "delimiterChar" : ",",
                  "endOfLineSymbol" : "\n"
                }, {
                  "@type" : "type.googleapis.com/RedactOperationConfig"
                } ]
  5. Set the needed hasHeader, quoteChar, columnIndices, delimiterChar and endOfLineSymbol .

Kindly note that the locale used to specify the filter influences the representation of the characters and when you pass that stream into another context, those characters might not be read the same way. The dashboard looks like one form to us, but some of the content is handed off to a process that might have a different regex library (RE2, PCRE (PHP), ECMAscript )

Was this article helpful?
0 out of 0 found this helpful

Comments

0 comments

Please sign in to leave a comment.