CSV Processing Configuration Options

The CSV Processing Operation in Operation pipeline supports the following configuration:

  • Quote character

  • Delimiter character

  • End of Line Symbol

In order to apply this configuration, the following steps need to be done:

  1. Dashboard → Routes → SFTP → Manage



  2. Export the route

  3. Open the exported route in a text editor and find the operations tag:

  4. Set the following value for the operations tag:

              operations: |-
                [ {
                  "@type" : "type.googleapis.com/ProcessInputStreamPairOperationConfig"
                }, {
                  "@type" : "type.googleapis.com/ReplaceCsvColumnOperationConfig",
                  "hasHeader" : true,
                  "quoteChar" : "`",
                  "columnIndices" : [ 3 ],
                  "delimiterChar" : ",",
                  "endOfLineSymbol" : "\n"
                }, {
                  "@type" : "type.googleapis.com/RedactOperationConfig"
                } ]
  5. Set the needed hasHeader, quoteChar, columnIndices, delimiterChar and endOfLineSymbol .

Kindly note that the locale used to specify the filter influences the representation of the characters and when you pass that stream into another context, those characters might not be read the same way. The dashboard looks like one form to us, but some of the content is handed off to a process that might have a different regex library (RE2, PCRE (PHP), ECMAscript )

Was this article helpful?
0 out of 0 found this helpful



Please sign in to leave a comment.