Processing a list of files

Post any question you may have in regards to GoAnywhere Director and let our talented support staff and other users assist you.

Processing a list of files

Postby MichaelB » Fri Feb 11, 2011 9:23 am

Greetings,

I would like to know the best method for processing a static list of files.

I have 8 files that I need to work on. The file names are always the same.

1) Retrieve 8 files from local folder - verify they all exist
2) SFTP transmit all 8 files to existing resource
3) Rename all files by adding a date prefix to each existing file name
4) Move all files to an local archive folder

Is the best way to process each file one by one or can I set an array type variable to loop through?

Thank you,
MichaelB
 
Posts: 1
Joined: Fri Feb 11, 2011 9:02 am

Re: Processing a list of files

Postby RElliott63 » Wed Jul 06, 2011 11:33 am

I know this is an old post from FEB but, here's a solution in case someone needs help with something similar... (*DISCLAIMER* --- Please test before using!)

Personally, I would keep the list of "Static" files in a table then read the table as needed. Then, if I ever needed to change a file name or add/delete a file, I only have one thing to change -- the table. Also, Renaming/Archiving the file can happen in the same statement (See <MOVE> below).

This also assumes that if the job errors on a particular file, it leaves the file as-is to process later.

Code: Select all
<project name="FTP Static Files" mainModule="Main" version="1.0" logLevel="debug" >

  <module name="Main" dependsOn="Get File List, Transmit">

      <deleteWorkspace label="Remove Workspace" version="1.0" />

   </module>                                                             

   <module name="Get File List" >

      <createWorkspace label="Create Workspace" version="1.0" />

      <timestamp label="Set TimeStamp" version="1.0">
         <format outputVariable="TimeStamp"    pattern="yyyyMMdd.hhmmss" />
      </timestamp>

      <setVariable label="Set HaveFiles" version="1.0" name="HaveFiles" value="Yes"/>

    <setVariable label="Set Error" version="1.0" name="Error" value="0" />
      <sql label="Connect to Server" version="1.0" resourceId="${Server}" onError="setVariable:Error=1" >
         <query label="Get File List" whenNoDataFound="error" outputVariable="FileList">
            <statement>
Select FileName
  From MyLib.MyFiles
        </statement>
         </query>
    </sql>

      <setVariable label="Set HaveFiles" version="1.0" name="HaveFiles" value="No" executeOnlyIf="${Error} gt 0"/>

  </module>

   <module name="Transmit" executeOnlyIf="'${HaveFiles}' eq 'Yes'">

    <forEachLoop label="Process each File" itemsVariable="${FileList}" currentItemVariable="ThisFile">

          <setVariable label="Set File" version="2.0" name="File" value="${ThisFile[1]}" />       

          <print label="Status" version="1.0">
              <![CDATA[ 
*================================================*
*    Processing:  ${File}     
*================================================*
          ]]>
        </print>

          <setVariable label="Set SFTPFile" version="1.0" name="SFTPFile" value="${DIRLocation}\${File}"/>
          <setVariable label="Set ARCHFile" version="1.0" name="ARCPFile" value="${ARCLocation}\${File}"/>

        <setVariable label="Set Error" version="1.0" name="Error" value="0" />
           <sftp label="Connect to Resource" version="1.0" resourceId="${SFTPResource}" onError="setVariable:Error=1">
             <put label="Put file" sourceFile="${SFTPFile}" destinationFile="${File}" destinationDir="${DESTLocation}" type="binary" />
          </sftp>

        <setVariable label="Set Status" version="1.0" name="Status" value="** ERROR ** Check prior messages and Retry" executeOnlyIf="${Error} gt 0"/>
        <setVariable label="Set Status" version="1.0" name="Status" value="** Job Successful **"                       executeOnlyIf="${Error} eq 0"/>

        <print label="File Status" version="1.0">
          <![CDATA[ 
*================================================*
  ${Status}
  SFTP'ing ${File} to ${SFTPResource}
*================================================*
          ]]>
           </print>

        <!-- Move/Rename file to Archive -->
        <move label="Move File" sourceFilesVariable="${SFTPFile}" destDir="${ARCHFile}" suffix=".${TimeStamp}" version="1.0" executeOnlyIf="${Error} eq 0"/>

    </forEachLoop>

   </module>

  <!-- Job Variables -->
   <variable name="Server"       value="ServerName"     description="iSeries Server or equivilent that holds file with list of file names to SFTP" />
   <variable name="SFTPResource" value="ResourceID"     description="Enter the ResourceID created for SFTP Communications" />
   <variable name="DIRLocation"  value="/FileDirectory" description="Directory Location where files exist" />
   <variable name="ARCLocation"  value="/ArchDirectory" description="Directory Location for Archiving file after Transmission" />
   <variable name="DESTLocation" value="/DestDirectory" description="Directory Location for Destination of file during SFTP" />
                                                                     
   <description>Process Outbound files</description>

</project>
RElliott63
 
Posts: 14
Joined: Thu Jul 01, 2010 10:42 am


Return to Community Forum

Who is online

Users browsing this forum: No registered users and 0 guests

cron