Directory Poller

Category: Kepler Actors — Other products by this author

Screenshot — click to enlarge

Project Description

Project resources

• Releases
• Issue tracker
• Contact address
• Code repository

Introduction

The Directory Poller actor watches a specified directory for new and modified files.

When new or modified files are found in the directory, for each file found a StringToken (containing the file's full path name) is sent onto the output port.

There is also a SRB Directory Poller (see Kepler SRB Actors) which preforms the same task on an SRB filesystem and uses the Jargon API's equivalents to the standard java API for file access.

How it works

• uses java.io.File#listFiles() to grab a listing of the directory as an array of java.io.File
• check each element in the array for the following:
• make sure it is a file (using java.io.File#isFile())
• check the last modified time of the file (using java.io.File#lastModified()) and calculate how long it has been since the file was last modified
• check the java.util.Hashtable (which stores previously found files) to make sure the file hasn't already been found
  
• if the file hasn't been modified for at least 5 seconds and is new or has been modified since it was previously found then add it to a java.util.ArrayList
• once each file in the directory has been checked then
• update the java.util.Hashtable to keep track of the newly found files.
• return the java.util.ArrayList of files

Issues

Sometimes files that are still being written to the filesystem can be detected before they finish writing. This is the reason why files must have been unmodified for at least 5 seconds before we count it as a new or modified file.

Unforutunatly this isn't a fool proof solution. There may be problems when high latency networks are being used to transfer files and other similar situations, especially when using the SRB Directory Poller.