Data Editing using SPIDR FlowCom Software

By Chad Cluver | Wed, 10 Mar 2010

The SPIDR gauge is the most robust, accurate wellhead pressure gauge on the market, and can be used to record high resolution data at high frequency (one data point per second) for up to 5 million data points.  Obviously, for extended duration tests this could lead to large volumes of data to work with, and large data files which could result in slow processing time.  Fortunately the SPIDR system utilizes a proprietary file structure that greatly compresses the data to allow for quick processing and small file size.  However when one wants to then take this data and utilize another commercially available analysis software package, it is required to then export the SPIDR data as an ascii file and then import into said software.  The resulting ASCII file is not compressed in any way, and as such can get increasingly large as test durations grow.  Fortunately the SPIDR FlowCom software, which is freely available for download from our website (FlowCom Download) has several tools to help trim large volumes of data down to more manageable sizes while still retaining the important characteristics to achieve meaningful results from analysis.  After the data has been trimmed to a more manageable size, it can then be exported as an ASCII file for use in any software package.


The most basic form of data editing is deleting unnecessary data.  This is easily done by accessing the “Edit” menu, as shown in Image 1 below and selecting “Delete Records”.  All data editing is done through this menu.  It is important to note, all editing functions reference the record number (green column), not the reading number.  It is also important to distinguish between “Reading” and “Record”.   “Reading” is the number attached to the sample and is never changed.  When a file is edited, the number of records in the file may change but the reading number associated with each remaining sample stays the same.  If the two numbers don’t match, then we know the file has been edited in some way.  The delete records option allows you to either delete a single data point, or larger blocks of data.  After “Delete Records” is selected, the data editing tool pops up, as shown in Image 2 below.  This same tool is used for all data editing functions.  The condition “Delete All” will be selected; this simply means all records in the range specified will be deleted.  It is important to note that the other data editing functions can be quickly accessed here as well.  Simply specify the starting record number and ending record number, and all data in between those records, including the start and end records, will be deleted.  This is especially useful to remove atmospheric pressure readings at the start and end of a file, which may be present if the job had a delayed start or if the gauge wasn’t stopped soon after removal from the well.

Image 2) Data Editing Tool

Another useful form of data editing is the “Must Go Up/Down” function.  These functions were developed for situations in which leaks at the well-head resulted in pressure spikes in the data.  They are especially useful for PBU’s/Draw-downs where it is expected that the pressure will be continually building/decreasing.  First identify at which record number in the data file to invoke the function, and where to end the function.  Then select the “Must Go Up” or “Must Go Down” function.  The data editing tool will pop up as it did before for the delete records option.  Now simply enter the points as determined earlier.  The program will then start at the specified start record and delete any reading which is less than/more than (depending on which function was chosen) the previous reading until it reaches the specified end point.  This is an easy way to remove noisy, erroneous data from the critical part of the test.


The “Filter Less Than/Greater Than” editing tool is another great way to remove an erroneous or extraneous data such as atmospheric pressure readings, or noise spikes that may occur from mechanical interference.  This function works in much the same way as the previous two, only this time the third input, the key value, is also required.  Simply determine the range of the data to be edited by selecting a starting and ending record number, and then enter the key value (pressure).  Any data above or below that value in the range specified will be deleted, depending upon which function was selected.


Probably the most useful data editing option, the “Condense Records” function, is also the most advanced offered in the FlowCom software.  It is a method to intelligently condense the number of records down to a more manageable size while still retaining the necessary detail during the critical parts of the test to achieve accurate analysis results.  The “Condense Records” function in FlowCom reduces the number of records in a file by deleting those readings which do not change significantly from adjacent readings.   Like all the data editing options, a range of data over which to invoke the function is entered.  Then a pressure window is entered, the pressure window being the amount of change between readings to compare.  Typically, one or two psi is used for the pressure window.  The “Condense Records” function will start with the first sample in the selected range and compare it to the next sample.  If the pressure change is not greater than plus or minus the pressure window, that sample is discarded.  This process continues until a sample is found whose change does exceed the pressure window.  The newly saved sample then becomes the basis for comparison and the process is repeated until the end of the range is reached.  When the “Condense Records” routine is completed, a window will pop up giving the number of samples at the start of the “Condense Records” process and the number at the end of the process and gives the user the option to repeat or reverse the process.  The user may then enter a new pressure window, either larger or smaller than the original pressure window depending on whether the “Condense Records” process resulted in too few or too many samples.  The “Condense Records” function allows for high data density when needed, such as when pressures are more rapidly changing (like at start of a PBU or DFIT fall-off), and much lower data density when needed such as when the pressures have stabilized (at the end of the PBU or fall-off).


The final data editing option is the “Mow the Lawn” function. This editing function was developed for the situation in which a data file is extremely noisy.  Typically, a noisy data file will have an overly large number of readings for a short period of time. The “Mow the Lawn” function works much like the condense function, a range of data to apply the function to is specified, and a key value (pressure window) is also specified.  The logic behind this editing function is to look for and delete the data extremes over a span of samples.  For example, if a plot of the data shows a pressure band that is five psi in magnitude, the user might enter a pressure window of 3 psi.  Starting with the first sample in the selected range, the program will then look at the next two samples to see if the changes are greater than 3 psi from the first sample.  If both samples are greater than 3 psi different, they are saved.  If only the first sample is greater than 3 psi, then that sample is considered a “spike” and it is discarded.  The process is repeated until reaching the end of the selected range after which the user will be prompted to condense again.  Subsequent condensation can use larger or smaller windows depending in the event the routine over or under reduced the desired effect.


After successfully editing the data to suit, that data will need to be exported in ASCII format so that it can be brought in to a commercially available analysis software package.  There are two options in FlowCom that allow for exporting the data in ASCII format.  The first is the “Make Text File” function, shown below in Image 3.  This will generate a text file that includes header information.  A sample is shown below in Image 4.  This is a standard ASCII text file, and may be imported directly into many software packages, or edited further in any spreadsheet software.  The header contains information such as the wellname, when the data starts and ends, the number of records, the serial number of the SPIDR used to capture the data, the file name, the sample window (the SPIDR can utilize intelligent recording that works much the same as the condense records option described above), the sample rate, and the maximum interval between samples.  The date and time are left as US calendar date and 24hr time formats in this text file.  Some software has a difficult time picking up the data and time as one column of data, and as such we also offer the ability to generate Delta Time ASCII files.  The “Make Delta Time” option, shown below in Image 5 will generate a basic ASCII file, with no header information, containing only a Delta Time (hours) column and the wellhead pressure column.  This can be seen below in Image 6.  This is a quick way to generate a delta time file, and saves the user from having to manually compute a delta time column before importing into an analysis package.

As can be seen, the freely available FlowCom software allows for easily editing data and quickly exporting that data for use in most any commercially available software analysis package.  This coupled with the high quality, high frequency data recorded by the SPIDR gauge allows for the best pressure data gathering system in the oil and gas industry.  We have expert staff available 24 hours a day, 7 days a week to help with any data processing needs, in addition to being able to answer any technical questions that may arise during your test.