datalad.cmd

Wrapper for command and function calls, allowing for dry runs and output handling

class datalad.cmd.GitRunner(*args, **kwargs)[source]

Bases: datalad.cmd.Runner

Runner to be used to run git and git annex commands

Overloads the runner class to check & update GIT_DIR and GIT_WORK_TREE environment variables set to the absolute path if is defined and is relative path

static get_git_environ_adjusted(env=None)[source]

Replaces GIT_DIR and GIT_WORK_TREE with absolute paths if relative path and defined

run(cmd, env=None, *args, **kwargs)[source]

Runs the command cmd using shell.

In case of dry-mode cmd is just added to commands and it is actually executed otherwise. Allows for separately logging stdout and stderr or streaming it to system’s stdout or stderr respectively.

Note: Using a string as cmd and shell=True allows for piping,
multiple commands, etc., but that implies shlex.split() is not used. This is considered to be a security hazard. So be careful with input.
Parameters:
  • cmd (str, list) – String (or list) defining the command call. No shell is used if cmd is specified as a list
  • log_stdout (bool, optional) – If True, stdout is logged. Goes to sys.stdout otherwise.
  • log_stderr (bool, optional) – If True, stderr is logged. Goes to sys.stderr otherwise.
  • log_online (bool, optional) – Whether to log as output comes in. Setting to True is preferable for running user-invoked actions to provide timely output
  • expect_stderr (bool, optional) – Normally, having stderr output is a signal of a problem and thus it gets logged at level 11. But some utilities, e.g. wget, use stderr for their progress output. Whenever such output is expected, set it to True and output will be logged at level 9 unless exit status is non-0 (in non-online mode only, in online – would log at 9)
  • expect_fail (bool, optional) – Normally, if command exits with non-0 status, it is considered an error and logged at level 11 (above DEBUG). But if the call intended for checking routine, such messages are usually not needed, thus it will be logged at level 9.
  • cwd (string, optional) – Directory under which run the command (passed to Popen)
  • env (string, optional) – Custom environment to pass
  • shell (bool, optional) – Run command in a shell. If not specified, then it runs in a shell only if command is specified as a string (not a list)
  • stdin (file descriptor) – input stream to connect to stdin of the process.
Returns:

Return type:

(stdout, stderr) - bytes!

Raises:

CommandError – if command’s exitcode wasn’t 0 or None. exitcode is passed to CommandError’s code-field. Command’s stdout and stderr are stored in CommandError’s stdout and stderr fields respectively.

class datalad.cmd.Runner(cwd=None, env=None, protocol=None, log_outputs=None)[source]

Bases: object

Provides a wrapper for calling functions and commands.

An object of this class provides a methods that calls shell commands or python functions, allowing for protocolling the calls and output handling.

Outputs (stdout and stderr) can be either logged or streamed to system’s stdout/stderr during execution. This can be enabled or disabled for both of them independently. Additionally, a protocol object can be a used with the Runner. Such a protocol has to implement datalad.support.protocol.ProtocolInterface, is able to record calls and allows for dry runs.

call(f, *args, **kwargs)[source]

Helper to unify collection of logging all “dry” actions.

Calls f if Runner-object is not in dry-mode. Adds f along with its arguments to commands otherwise.

Parameters:f (callable) –
commands
cwd
dry
env
log(msg, *args, **kwargs)[source]

log helper

Logs at level 9 by default and adds “Protocol:”-prefix in order to log the used protocol.

log_cwd
log_env
log_outputs
log_stdin
protocol
run(cmd, log_stdout=True, log_stderr=True, log_online=False, expect_stderr=False, expect_fail=False, cwd=None, env=None, shell=None, stdin=None)[source]

Runs the command cmd using shell.

In case of dry-mode cmd is just added to commands and it is actually executed otherwise. Allows for separately logging stdout and stderr or streaming it to system’s stdout or stderr respectively.

Note: Using a string as cmd and shell=True allows for piping,
multiple commands, etc., but that implies shlex.split() is not used. This is considered to be a security hazard. So be careful with input.
Parameters:
  • cmd (str, list) – String (or list) defining the command call. No shell is used if cmd is specified as a list
  • log_stdout (bool, optional) – If True, stdout is logged. Goes to sys.stdout otherwise.
  • log_stderr (bool, optional) – If True, stderr is logged. Goes to sys.stderr otherwise.
  • log_online (bool, optional) – Whether to log as output comes in. Setting to True is preferable for running user-invoked actions to provide timely output
  • expect_stderr (bool, optional) – Normally, having stderr output is a signal of a problem and thus it gets logged at level 11. But some utilities, e.g. wget, use stderr for their progress output. Whenever such output is expected, set it to True and output will be logged at level 9 unless exit status is non-0 (in non-online mode only, in online – would log at 9)
  • expect_fail (bool, optional) – Normally, if command exits with non-0 status, it is considered an error and logged at level 11 (above DEBUG). But if the call intended for checking routine, such messages are usually not needed, thus it will be logged at level 9.
  • cwd (string, optional) – Directory under which run the command (passed to Popen)
  • env (string, optional) – Custom environment to pass
  • shell (bool, optional) – Run command in a shell. If not specified, then it runs in a shell only if command is specified as a string (not a list)
  • stdin (file descriptor) – input stream to connect to stdin of the process.
Returns:

Return type:

(stdout, stderr) - bytes!

Raises:

CommandError – if command’s exitcode wasn’t 0 or None. exitcode is passed to CommandError’s code-field. Command’s stdout and stderr are stored in CommandError’s stdout and stderr fields respectively.

datalad.cmd.get_runner(*args, **kwargs)[source]

Just a little helper to hardlink files’s load