Run TUFLOW FV From A Batch File
TUFLOW simulations can also be started from a batch file. Batch files are text files which contain a series of commands. Batch files allow a large degree of flexibility in starting TUFLOW simulations, advanced batch files can be used to:
- Run multiple simulations
- Set CPU priority for TUFLOW simulations
- Create a wait (e.g. start a simulation in 1 hour)
- Test simulation input
- Post process results after simulation has finished.
Batch files are executed by double clicking the created .bat file from Windows Explorer.
- Lowering the priority of a simulation to allow you to continue working;
- Setting a high priority for critical runs; or
- Making sure you runs are high priority on a shared computing resource!
- /LOW
- /BELOWNORMAL
- /NORMAL
- /ABOVENORMAL
- /HIGH
- /REALTIME
- M01_Q050_24hr_2m_Dev_001.fvc; and
- M01_Q100_24hr_2m_Dev_001.fvc;
- -e 10%
- -e 10pct
- -e 10%% (TUFLOW will search for 10%)
Simple batch file
To use a batch file, create a new text file in the TUFLOWFV\runs directory. The name is arbitrary but the file must have the extension ".bat". For example the file name could be _run_TUFLOW.FVbat.
Open this file in a text editor and add the following line.
"C:\TUFLOWFV\Releases\2023.1\TUFLOWFV.exe" "M01_5m_001.fvc"
This is the simplest command line for starting TUFLOW, the line above contains only two arguments, the first is the path to the TUFLOW FV executable and the second argument "M01_5m_001.fvc" is the simulation file name. Depending on the location of the executable the above command line will need to be modified. The quotations around the TUFLOW executable and the .fvc file name are only required if there are spaces in the file names. However, it is recommended to include these to avoid problems.
Batch File Switches
These switches are available in all batch files, they are not specific to TUFLOW.
Pause
A second line with the text "pause" can be added, this will prompt the user to press a key at the end of the batch file. This is useful if the DOS window disappears from the screen, before you can read the output. The pause command will keep the window open.
"C:\TUFLOWFV\Releases\2023.1\TUFLOWFV.exe" "M01_5m_001.fvc" pause
Comments
Inserting a comment or remark using REM at the start of a line will ignore execution. This is useful when choosing which runs to execute or to add user information to the batch file.
REM Existing Conditions "C:\TUFLOWFV\Releases\2023.1\TUFLOWFV.exe" "M01_5m_001.fvc" REM "C:\TUFLOWFV\Releases\2023.1\TUFLOWFV.exe" "M01_5m_002.fvc" pause
CPU parallelisation
The number of logical CPU processors available to a TUFLOW FV simulation can be limited via the OMP_NUM_THREADS environment variable. In the example below, components of the TUFLOW FV simulation that use parralell CPU processing will be limited to 3 logical processors. The default is to use all available processors if the TUFLOW FV license permits this number.
set OMP_NUM_THREADS=3 "C:\TUFLOWFV\Releases\2023.1\TUFLOWFV.exe" "M01_5m_001.fvc" pause
Starting process in new window
In order to run multiple simulations at the same time, each simulation needs be started in a new console window. To do this begin the batch file line with the "Start" command, a for the window is then added. For example, to start the simulation above in a new window titled "TUFLOWFV" the following would be used.
Start "TUFLOWFV" "C:\TUFLOWFV\Releases\2023.1\TUFLOWFV.exe" "M01_5m_001.fvc"
Wait
If multiple simulations are to be run, it is often desirable to run these in series, i.e. the second simulation starts after the first has finished. a /wait switch can be added which makes the batch file wait until the process is finished before moving onto the next command.
An example of this is:
Start "TUFLOWFV" /wait "C:\TUFLOWFV\Releases\2023.1\TUFLOWFV.exe" M01_5m_001.fvc Start "TUFLOWFV" /wait "C:\TUFLOWFV\Releases\2023.1\TUFLOWFV.exe" M01_2.5m_001.fvc
Without the wait command, both simulations would start at the same time.
Note: Unless there is a third simulation in the batch file the /wait command in the second line is not required, this prevents the batchfile from moving onto the third line.
Setting a priority
The priority of a process (such a TUFLOW simulation) can be specified within a batch file. This can be useful for a number of reasons:
The following are valid priority switches (in increasing order of priority):
An example of a low priority simulation is:
start "TUFLOWFV" /low "C:\TUFLOWFV\Releases\2023.1\TUFLOWFV.exe" "M01_5m_001.fvc"
TIP: The priority of a TUFLOW simulation can be changed at a later stage by using the Windows Task Manager, navigate to the Processes tab and right click on the TUFLOW process (e.g. TUFLOW_iSP_w64.exe) and select the set priority option.
Minimising the console window
Using the /min switch will keep the console window for TUFLOW models minimised, so it doesn't pop up to the front every time when the next model starts.
Start "TUFLOWFV" /wait /min "C:\TUFLOWFV\Releases\2023.1\TUFLOWFV.exe" M01_5m_001.fvc Start "TUFLOWFV" /wait /min "C:\TUFLOWFV\Releases\2023.1\TUFLOWFV.exe" M01_2.5m_001.fvc
TUFLOW switches in a batch file
These switches are specific to TUFLOW simulations.
Testing a simulation
The -t (test) switch is very useful for testing the data input without running the simulation. It is good practice to use this switch before carrying out the simulations, as this will tell you whether there are any data input problems. The -t switch runs TUFLOW to just before it starts the hydrodynamic computations.
An example is:
start "TUFLOWFV" "C:\TUFLOWFV\Releases\2023.1\TUFLOWFV.exe" -t "M01_5m_001.fvc"
Batch mode in TUFLOW
Unlike TUFLOW Classic/HPC, TUFLOW FV is run in batch mode by default. There is no requirement for a -b switch, and this switch is not supported by TUFLOW FV.
GPU Module Control
The -pu switch can be used to select which GPU card to direct the simulation towards. For example, to direct the simulation to GPU devices 0, specify -pu0. (Note: the GPU device numbering starts a 0, not 1). For example:
start "TUFLOWFV" "C:\TUFLOWFV\Releases\2023.1\TUFLOWFV.exe" -pu0 "C:\TUFLOWFV\Tutorial_Wiki\TUFLOWFV\runs\M01_5m_001.fvc"
Event and Scenario Management
Available within the 2024.0 TUFLOW FV Release, the -e and -s switches can be used to manage multiple events and scenarios from a single TUFLOW FV control file. The -e and -s wildcard is replaced by the following entry specified in the batch file. For example, the following batch file commands will run two simulations in series:
Start "TUFLOWFV" /wait "TUFLOWFV" "C:\TUFLOWFV\Releases\2024.0\TUFLOWFV.exe" -e1 Q050 -e2 24hr -s1 2m -s2 Dev "C:\TUFLOW\Tutorial_Wiki\TUFLOWFV\runs\M01_~e1~_~e2~_~s1~_s2~_001.fvc" Start "TUFLOWFV" /wait "TUFLOWFV" "C:\TUFLOWFV\Releases\2024.0\TUFLOWFV.exe" -e1 Q100 -e2 24hr -s1 2m -s2 Dev "C:\TUFLOWFV\Tutorial_Wiki\TUFLOWFV\runs\M01_~e1~_~e2~_~s1~_s2~_001.fvc"
Please be aware that the % symbol is a special character in a batch file. If the % symbol is used within a wildcard, it will not be read when searching for that wildcard in TUFLOW.
For example if the following event wildcard is used;
TUFLOW will search for '10' instead of '10%' as the event wildcard.
It is recommended that modellers avoid the use of a % symbol in any wildcard values. The string 'pct' is a great alternative for the % symbol.
For example, using:
Alternatively, using the % symbol twice in a row as shown below should result in a desired wildcard value.
TUFLOW FV Environment Variables
As of TUFLOW FV 2023.1 batch file environment variables can be passed to TUFLOW FV. For example, to set the location of the output directory via the environment variable OUTPUT_DIR:
set OUTPUT_DIR=..\results\run_007 start "TUFLOWFV" "C:\TUFLOWFV\Releases\2023.1\TUFLOWFV.exe" "C:\TUFLOWFV\Tutorial_Wiki\TUFLOWFV\runs\M01_5m_001.fvc"
The environment variable OUTPUT_DIR is referenced within the fvc using the $ syntax as follows:
At runtime, TUFLOW FV dynamically replaces $OUTPUT_DIR with the value assigned to OUTPUT_DIR in the batch file, in this case ..\results\run_007. This use of batch environment variables can be applied to any command within the fvc.
Advanced Batch Files
The below section shows example batch files for maximizing run efficiency,
Creating a delay
Windows 7 and newer
For a Windows 7 and newer operating system a delay of a set duration can be created with the Timeout <seconds> command. For example:
Timeout 10
Will create a 10 second delay, a countdown timer is given. Press any key to continue.
Other Systems
It is possible to create a delay of specified length in a batch file. The best way do this is actually to create a second batch file! To do this create a text file and call it wait.bat. Within that file enter the following text:
@ping 127.0.0.1 -n 2 -w 1000 > nul @ping 127.0.0.1 -n %1% -w 1000> nul
To create the pause we ping an address for a set time and disregard the output. Once the wait.bat has been created, it can be called from another batchfile using the following syntax:
call wait <time in seconds.
In the example below, the first simulation is started and then 60 seconds later (using the wait.bat file) the second simulation is started. Note there is no /wait flag specified, if there was the batchfile would wait for 60 seconds after the first simulation had finished before starting the second simulation.
Start "TUFLOWFV" "C:\TUFLOWFV\Releases\2023.1\TUFLOWFV.exe" M01_5m_001.fvc call wait 60 Start "TUFLOWFV" /wait "C:\TUFLOWFV\Releases\2023.1\TUFLOWFV.exe" M01_2.5m_001.fvc
For this to work the wait.bat file needs to be in the same folder as the batchfile calling it. You can put the file in a specific location and add this path as an environment variable. If this is done "call wait
Variables
Batch files can be easily setup so that they are more generic and easily customised when moving from one project to another. For example, in the below a variable, TUFLOWEXE, is used to define the path to the TUFLOW exe to be used, and a variable RUN is used to incorporate options such as the /wait so that the simulations run in series (one after the other).
set TUFLOWEXE=C:\TUFLOWFV\Releases\2023.1\TUFLOWFV.exe set RUN=start "TUFLOWFV" /wait "%TUFLOWEXE%" %RUN% MR_H99_C25_Q100.fvc %RUN% MR_H99_C25_Q050.fvc %RUN% MR_H99_C25_Q020.fvc
The advantage of using variables is if the path to the TUFLOW exe changes, or to run a different version of TUFLOW, it is just a simple change in the .bat file. In the above, note the use of quotes around %TUFLOWEXE% in the definition for the RUN variable – quotes are needed around file pathnames whenever they contain a space.
Looping in a batch file
If you have a large number of events to run using the event files a for loop can be used in a batch file. In this example we have 5 event magnitudes (-e1 input) and 5 event durations (-e2 input). The following batch file can be used to run all 25 simulations one after the other:
@echo off
REM This sets the variables as local, so you can use another batch file with A and B variables
SetLocal
REM set up variables
set A=Q010 Q020 Q050 Q100 Q200
set B=10min 30min 60min 120min 270min
REM Loop Through
FOR %%a in (%A%) do (
FOR %%b in (%B%) DO (
start "TUFLOWFV" /wait C:\TUFLOWFV\Releases\2023.1\TUFLOWFV.exe -e1 %%a -e2 %%b filename.fvc
)
)
pause
In order to test the syntax, you can test the batch file by replacing the following line:
start "TUFLOWFV" /wait "C:\TUFLOWFV\Releases\2023.1\TUFLOWFV.exe" -e1 %%a -e2 %%b filename.fvc
With the following line:
echo start "TUFLOWFV" /wait C:\TUFLOWFV\Releases\2023.1\TUFLOWFV.exe -e1 %%a -e2 %%b filename.fvc
By using the echo in the command line, rather than starting the simulations the command line will be displayed in the DOS output. In the example above the DOS output is shown in the image below (an analogous TUFLOW Classic console output example).
Alternatively this can be achieved using the following syntax, which is essentially the same but has the for loops compressed to a single line:
@echo off setlocal set A=Q010 Q020 Q050 Q100 Q200 set B=10min 30min 60min 120min 270min FOR %%a in (%A%) do FOR %%b in (%B%) do start "TUFLOWFV" "C:\TUFLOWFV\Releases\2023.1\TUFLOWFV.exe" -e1 %%a -e2 %%b filename_~e1~_~e2~.fvc pause
This can be extended to more variables for example:
@echo off
setlocal
set A=Q010 Q020 Q050 Q100 Q200
set B=10min 30min 60min 120min 270min
set C=Exg Dev Mit
FOR %%a in (%A%) do (
FOR %%b in (%B%) do (
FOR %%c in (%C%) do (
start "TUFLOWFV" /wait "C:\TUFLOWFV\Releases\2023.1\TUFLOWFV.exe" -e1 %%a -e2 %%b -s1 %%c filename_~e1~_~e2~_~s1~.fvc
)
)
)
pause
Looping in a batch file up to a certain number of runs
The below example shows a method for controlling the number of concurrent simulations.
As a brief description of the below, the "do_while_loop_start" counts the number of processes in the task manager which have "TUFLOWFV" in the name. If this is greater than or equal to the variable "CPU_Cores" then the batch file waits for 60 seconds.
@ echo off
Set A=Q001 Q002 Q005 Q010 Q020 Q050 Q100
Set B=Exg Dev
set /a CPU_Cores=3
FOR %%a in (%A%) DO (
FOR %%b in (%B%) DO (
call :do_while_loop_start
Start "TUFLOWFV" "C:\TUFLOWFV\Releases\2023.1\TUFLOWFV.exe" -e1 %%a -s1 %%b M01_5m_~e1~_~s1~_001.fvc
)
)
Pause
:do_while_loop_start
set /a count=0
for /f %%x in ('tasklist ^| find /c "TUFLOWFV"') do set count=%%x
if %count% geq %CPU_Cores% (
PING 1.1.1.1 -n 1 -w 60000 >NUL
goto do_while_loop_start
)
Another way to achieve the same result using IF logic is shown below. This batch file will loop through the series of 14 simulations in groups of three. The batch file will pause until the last of the three active model simulations are complete. When complete, the next series of three simulations will be started in parallel.
@ echo off
setlocal enabledelayedexpansion
SET A=Q001 Q002 Q005 Q010 Q020 Q050 Q100
SET B=Exg Dev
SET P=3
SET /A Counter=0
FOR %%a in (%A%) DO (
FOR %%b in (%B%) DO (
SET /A Counter+=1
IF /i !Counter!==!P! (
Start "TUFLOWFV" /wait "C:\TUFLOWFV\Releases\2023.1\TUFLOWFV.exe" -e1 %%a -s1 %%b M01_5m_~e1~_~s1~_001.fvc
SET /A Counter=0
) ELSE (
Start "TUFLOWFV" "C:\TUFLOWFV\Releases\2023.1\TUFLOWFV.exe" -e1 %%a -s1 %%b M01_5m_~e1~_~s1~_001.fvc
)
)
)
Looping in a batch file using scenario logic
Sometimes when using different scenarios, it is useful to run a specific set of events for one scenario but other events for a different scenario.
This can be automated within looped batch files using a bit of logic.
In the the example below there is model with defended and undefended scenarios. The defended scenario needs to be run for the 20, 100 and 200 year events but the undefended scenario is only run for the 100 year event. All models are run for the hr and 3hr critical durations.
The below batch file also uses the looping up to a certain number of runs, as seen in the point above.
To customize this script, alter the location of the TUFLOW executable, the name of fvc and the scenarios that you would like to run (Refer to 'Set A==', 'Set C==' and 'Set B==').
@ echo off
setlocal enabledelayedexpansion
REM ______________SET RUN VARIABLES_____________
set TUFLOWEXE=C:\TUFLOWFV\Releases\2023.1\TUFLOWFV.exe
set RUN=start "TUFLOWFV" /low "%TUFLOWEXE%"
set /a CPU_Cores=5
set fvc=M01_5m_~s1~_~e1~_~e2~_001.fvc
Set A=UDEF DEF
REM set B in loop beow
Set C==01hr 03hr
REM ______________SET LOOPS____________________
FOR %%a in (%A%) do (
REM _____DEFINE RUN LOGIC______
IF "%%a"=="UDEF" (
set B=100yr
)
IF "%%a"=="DEF" (
set B=020yr 100yr 200yr
)
REM ______________RUN LOOPS____________________
FOR %%b in (!B!) do (
FOR %%c in (%C%) do (
call :do_while_loop_start
%RUN% -s1 %%a -e1 %%b -e2 %%c %fvc%
timeout 60
)
)
)
REM ___________COUNT RUN INSTANCES______________
:do_while_loop_start
set /a count=0
for /f %%x in ('tasklist ^| find /c "TUFLOWFV"') do set count=%%x
if %count% geq %CPU_Cores% (
PING 1.1.1.1 -n 1 -w 60000 >NUL
goto do_while_loop_start
)
endlocal
Coordinating Asynchronous Simulations
This batch file will set off asynchronous runs on separate threads, and then wait for all runs to finish before moving to the next process. This example kicks off 8 tuflowfv.exe processes and then waits before collating (in fortran) and processing the results (in python). Both “combine_therad_outputs.exe” and “Post_process.py” are dummy programs only (with dummy arguments), and are included just to demonstrate how any sort of post processing might be called after the synchronisation of all TUFLOW simulations.
@echo off :: Set local setlocal :: User defined variables – will need to be changed set "for_infile=inputs_fortran.inp" set "run_exe=tuflowfv.exe" set "file_prefix=C:\temp\test" set "sum_exe=combine_thread_outputs.exe" set "n_threads=8" set "pyt_code=Post_process.py" set "pyt_infile=inputs_python.inp" :: Random lock file set "lock=%temp%\wait%random%.lock" :: Launch processes asynchronously, with stream 9 redirected to a lock file. :: The lock file will remain locked until the individual exe instance ends. :: %%a is sent to %run_exe% as an argument to set the thread to use for /L %%a in (0,1,7) do ( start "" 9>"%lock%%%a" %run_exe% %for_infile% %%a ) :: Wait for the processes to finish :Waitrun 1>nul 2>nul ping /n %n_threads% ::1 for /L %%N in (0,1,7) do ( (call ) 9>"%lock%%%N" || goto :Waitrun ) 2>nul :: Delete the lock files del "%lock%*" :: Collate thread outputs %sum_exe% %file_prefix% %n_threads% :: Make image python %pyt_code% %pyt_infile% :: Finish up echo Done.
Shutdown on Completion
Often we may have a number of TUFLOW simulations going when we leave work on a Friday afternoon, if these are not going to take all weekend to run, using some of the batch file logic above you can create a batch file that will shutdown the computer when no TUFLOW simulations are running. An example batch file for this purpose is below:
echo off
:do_while_loop_start
set /a count=0
for /f %%x in ('tasklist ^| find /c "TUFLOWFV"') do set count=%%x
IF %count%==0 (
shutdown.exe /s /t 30
) ELSE (
timeout 60
goto do_while_loop_start
)
pause
This batch file counts the number of processes with "TUFLOWFV" in the process name and if this is greater than 0, a delay of 60 seconds is created and the loop started again. If no "TUFLOWFV" simulations are running a shutdown command is issued.
To test the batch file it is recommended that you add an "echo" in front of the shutdown.exe /s /t 30 command to ensure it is working correctly (I use this on a Windows 7 machine, but have not tested it on either XP or Windows 8).
In order to restart the machine rather than power it off the /s can be replaced with /r. This can be useful when a heavily utilised modelling computer needs a restart!