Design

The chapter described command API principles and the design of particular subsystems in DataLad.

• Command line interface
  • Basic workflow of a command line based command execution
  • Command parser construction by Interface inspection
  • CLI-based execution of Interface command
• Provenance capture
  • The provenance record
  • Declaration of inputs and outputs
  • Placeholders in commands and IO specifications
  • Result records emitted by run
  • Implementation details
• Application-type vs. library-type usage
  • Library-mode implications
• File URL handling
  • Extensible protocol and authentication support
  • Indexing and access of archive content
• Result records
  • Mandatory fields
  • Common optional fields
  • Additional fields observed “in the wild”
• dataset argument
  • Impact on relative path resolution
  • Special values
  • Use cases
• Log levels
  • Common principles
  • Use cases
• Drop dataset components
  • Use cases
• Python import statements
  • Examples
• Miscellaneous patterns
  • Generator methods in Repo classes
  • Calls to Git commands
  • Command examples
• Exception handling
  • Catching exceptions
  • Messaging about an exception
• Credential management
  • Credentials
  • Providers
  • Integration with Git
  • Authenticators
• URL substitution
  • Examples
• Threaded runner
  • Threads
  • Protocols
  • Object and Generator Results
• BatchedCommand and BatchedAnnex
  • Batched Command
  • BatchedAnnex
• Standard parameters
• Positional vs Keyword parameters
  • Motivation
  • Interfaces
  • Regular functions and methods
• Docstrings
  • Formatting overview and guidelines
• Progress reporting
  • Design and implementation
  • Reporting progress with log_progress()
  • Reporting progress with with_(result_)progress()
  • Output non-progress information without interfering with progress bars
• GitHub Action
  • Example Usage
  • Options
• Continuous integration and testing
  • Running tests
  • Writing tests
  • Migrating tests from nose to pytest
• User messaging: result records vs exceptions vs logging
  • Motivation
  • Specification
  • Examples