Adding a field in python
currently I am using python to create a few directories.. what I would like to do is add another column to the data and put the path in it
currently:
FILES=self.inputs[0].read()
PTH=os.path.dirname(FILES["APV_CSV"])+"/"
if not os.path.exists(PTH+"Clean"):
os.makedirs(PTH+"Clean")
os.chmod(PTH+"Clean",0o777)
self.outputs[0].write(FILES)
id like to do something like this:
FILES=self.inputs[0].read()
FILES.append("PATH",str)
PTH=os.path.dirname(FILES["APV_CSV"])+"/"
FILES["PATH"]=PTH
am I close? lol
Thanks!
-
Yes, very close. The example above looks like Brainscript but the question is for Python. I'm assuming you'd like to do this in Analyze. This is the common syntax in Analyze for all the nodes that allow you to modify Python (e.g. Transform, Generate Data, Split, Filter, Aggregate, Joins, Lookups, etc...)
First add the output column in the Configure Fields section and then simply populate it with either a value or a variable in the Process Records section.
ConfigureFields
out1.path = unicode
ProcessRecords
out1.path = "mypath"
Attached files
-
Any particular reason you are using the "Python Node" vs "Generate Data" or the Transform node? The syntax for that node is slightly different as it's typically used for legacy dataflows/graphs. The new nodes are much simpler to use.
Regardless, here is the syntax for the legacy "Python Node" (make sure you are using tabs for indention)
To create the output column on the output pin use this:
pathName = "mypath" #Example variable
self.som = self.newMetadata() #Create output metadata
self.som.append("Path","string") #Create a new field called Path
self.outputs[0].metadata = self.som #Add metadata to the output pinTo populate use this:
outSchemaRec = self.outputs[0].newRecord() #Create a new output record
outSchemaRec[0] = pathName #Set the value of the new field
self.outputs[0].write(outSchemaRec) #Write the output record
-
I am trying to use the python node because its already being used to create the sub folders.. and the path and such are ready to go. Although yeah, i could just create the field in a transform node before this node...
but i just realized, its useless. The export nodes dont allow you to use a field name for the file, but the import does?
is there a way to set a parameter dynamically?
-
The typical, and simplest method, to accomplish this is to use a 2nd node to rename the file. Below is an alternate method that technically works, but is subject to change in future releases using a default run time property and a JEXL command.
FileName:
c:/MyTemp/TEST_{{^RunDate^}}.xlsx
or
c:/MyTemp/TEST_{{!jexl!parameters.get("RunDate").replace("-","").replace(":","").substring(0,4)!}}.xlsx
To insert a run property or see a list, simply click on the context menu for the property. e.g.
-
The preferred, and supported, method to parameterize the name is to use the path safe version of the RunDate property reference: {{^RunDate_PathSafe^}} to insert the date without needing to strip out the element delimiter characters in the date e.g.
C:/MyTemp/TEST_{{^RunDate_PathSafe^}}.xlsx
Which results in a file named Test_20200110.xlsx
-
Note the Run* run properties were first introduced in Analyze v.3.5.1
See the following topic in your Help documentation for further details:
Advanced topics > Using derived property values > Run property substitution
This information is also available online for the latest version of the product here:
-
Use the techniques described in this article to rename a file. You can create a Run Dependency so that the Transform node only runs once the output connector node has finished running.
For more information on Run Dependencies see the following topic in your Help documentation:
Advanced topics > Creating run dependencies
The information is also available online for the latest version of the product here:
Please sign in to leave a comment.
Comments
10 comments