gitoxide

gix is a command-line interface (CLI) to access git repositories in various ways best described as low-level for use by experts or those validating functionality in real-world scenarios. Performance and efficiency are staples of the implementation.

ein is reserved for one-off tools that are useful to many, and will one day implement a truly unique workflow with the potential to become the preferred way to interact with git repositories.

Please note that all functionality comes from the gitoxide-core library, which mirrors these capabilities and itself relies on all git-* crates. It's not meant for consumption, for application development, please use git-repository.

• the ein program - convenient and for humans
  • init - initialize a new non-bare repository with a main branch
  • clone - initialize a local copy of a remote repository
  • tools
    • organize - find all git repositories and place them in directories according to their remote paths
    • find - find all git repositories in a given directory - useful for tools like skim
    • estimate-hours - estimate the time invested into a repository by evaluating commit dates.
      • Based on the git-hours algorithm.
      • See the discussion for some performance data.
• the gix program (plumbing) - lower level commands for use in automation
  • progress - provide an overview of what works and what doesn't from the perspective of the git configuration. This is likely to change a lot over time depending on actual needs, but maybe useful for you to see if particular git-configuration is picked up and where it deviates.
  • config - list the complete git configuration in human-readable form and optionally filter sections by name.
  • exclude
    • query - check if path specs are excluded via gits exclusion rules like .gitignore.
  • verify - validate a whole repository, for now only the object database.
  • commit
    • describe - identify a commit by its closest tag in its past
  • tree
    • entries - list tree entries for a single tree or recursively
    • info - display tree statistics
  • odb
    • info - display odb statistics
    • entries - display all object ids in the object database
  • mailmap
    • entries - display all entries of the aggregated mailmap git would use for substitution
  • revision
    • list - list plain revision hashes from a starting point, similar to a very simple version of git rev-list.
    • explain - show what would be done while parsing a revision specification like HEAD~1
    • resolve - show which objects a revspec resolves to, similar to git rev-parse but faster and with much better error handling
    • previous-branches - list all previously checked out branches, powered by the ref-log.
  • remote
    • refs - list all references available on the remote based on the current remote configuration.
    • ref-map - show how remote references relate to their local tracking branches as mapped by refspecs.
  • fetch - fetch the current remote or the given one, optionally just as dry-run.
  • clone
    • initialize a new bare repository and fetch all objects.
    • initialize a new repository, fetch all objects and checkout the main worktree.
  • credential
    • fill/approve/reject - The same as git credential, but implemented in Rust, calling helpers only when from trusted configuration.
  • free - no git repository necessary
    • pack
      • verify
      • index verify including each object sha1 and statistics
      • explode, useful for transforming packs into loose objects for inspection or restoration
        • verify written objects (by reading them back from disk)
      • receive - receive a whole pack produced by pack-send or git-upload-pack, useful for clone like operations.
      • create - create a pack from given objects or tips of the commit graph.
      • send - create a pack and send it using the pack protocol to stdout, similar to 'git-upload-pack', for consumption by pack-receive or git-receive-pack
      • multi-index
        • info - print information about the file
        • create - create a multi-index from pack indices
        • verify - check the file for consistency
        • entries - list all entries of the file
      • index
        • create - create an index file by streaming a pack file as done during clone
          • support for thin packs (as needed for fetch/pull)
    • commit-graph
      • verify - assure that a commit-graph is consistent
    • mailmap
      • verify - check entries of a mailmap file for parse errors and display them
    • index
      • entries - show detailed entry information for human or machine consumption (via JSON)
      • verify - check the index for consistency
      • info - display general information about the index itself, with detailed extension information by default
        • detailed information about the TREE extension
        • …other extensions details aren't implemented yet
      • checkout-exclusive - a predecessor of git worktree, providing flexible options to evaluate checkout performance from an index and/or an object database.

git-actor

• read and write a signature that uniquely identifies an actor within a git repository
• a way to parse name <email> tuples (instead of full signatures) to facilitate parsing commit trailers.

git-hash

• types to represent hash digests to identify git objects.
• used to abstract over different kinds of hashes, like SHA1 and the upcoming SHA256
• API documentation
  • Some examples

git-chunk

• decode the chunk file table of contents and provide convenient API
• write the table of contents

git-hashtable

• hashmap
• hashset

git-object

• decode (zero-copy) borrowed objects
  • commit
    • parse trailers
  • tree
• encode owned objects
  • commit
  • tree
  • tag
    • [name validation][tagname-validation]
• transform borrowed to owned objects
• API documentation
  • Some examples

git-pack

• packs
  • traverse pack index
  • 'object' abstraction
    • decode (zero copy)
    • verify checksum
  • simple and fast pack traversal
    • fast pack traversal works with ref-deltas
  • decode
    • full objects
    • deltified objects
  • decode
    • decode a pack from Read input
      • Add support for zlib-ng for 20% faster decompression performance
      • Read to Iterator of entries
        • read as is, verify hash, and restore partial packs
    • create index from pack alone (much faster than git)
      • resolve 'thin' packs
  • encode
    • Add support for zlib-ng for 2.5x compression performance
    • objects to entries iterator
      • input objects as-is
      • pack only changed objects as derived from input
      • base object compression
      • delta compression
        • respect the delta=false attribute
      • create 'thin' pack, i.e. deltas that are based on objects the other side has.
      • parallel implementation that scales perfectly
    • entries to pack data iterator
    • write index along with the new pack
  • verify pack with statistics
    • brute force - less memory
    • indexed - optimal speed, but more memory
  • advanced
    • Multi-Pack index file (MIDX)
      • read
      • write
      • verify
    • 'bitmap' file
    • special handling for networked packs
    • detect and retry packed object reading
• API documentation
  • Some examples

git-odb

• loose object store
  • traverse
  • read
    • into memory
    • streaming
    • verify checksum
  • streaming write for blobs
  • buffer write for small in-memory objects/non-blobs to bring IO down to open-read-close == 3 syscalls
  • read object header (size + kind) without full decompression
• dynamic store
  • auto-refresh of on-disk state
  • handles alternates
  • multi-pack indices
  • perfect scaling with cores
  • support for pack caches, object caches and MRU for best per-thread performance.
  • prefix/short-id lookup, with optional listing of ambiguous objects.
  • object replacements (git replace)
  • high-speed packed object traversal without wasted CPU time
    • user defined filters
  • read object header (size + kind) without full decompression
• sink
  • write objects and obtain id
• alternates
  • resolve links between object databases
  • safe with cycles and recursive configurations
  • multi-line with comments and quotes
• promisor
  • It's vague, but these seems to be like index files allowing to fetch objects from a server on demand.
• API documentation
  • Some examples

git-diff

Check out the performance discussion as well.

• tree
  • changes needed to obtain other tree
  • case-insensitive comparisons
  • rename and copy tracking
  • readily available caching for 4x+ speedups
• patches
  • There are various ways to generate a patch from two blobs.
  • any
• lines
  • Simple line-by-line diffs powered by the imara-diff crate.
• diffing, merging, working with hunks of data
• find differences between various states, i.e. index, working tree, commit-tree
• API documentation
  • Examples

git-traverse

Check out the performance discussion as well.

• trees
  • nested traversal
• commits
  • ancestor graph traversal similar to git revlog
• API documentation
  • Examples

git-url

• As documented here: https://www.git-scm.com/docs/git-clone#_git_urls
• parse
  • ssh URLs and SCP like syntax
  • file, git, and SSH
  • paths (OS paths, without need for UTF-8)
• username expansion for ssh and git urls
• convert URL to string
• API documentation
  • Some examples

git-packetline

• PKT-Line
• encode
• decode (zero-copy)
• error line
• V2 additions
• side-band mode
• Read from packet line with (optional) progress support via sidebands
• Write with built-in packet line encoding
• async support
• API documentation
  • Some examples

git-transport

• No matter what we do here, timeouts must be supported to prevent hanging forever and to make interrupts destructor-safe.
• client
  • general purpose connect(…) for clients
    • file:// launches service application
    • ssh:// launches service application in a remote shell using ssh
    • git:// establishes a tcp connection to a git daemon
    • http(s):// establishes connections to web server
      • via curl (blocking only)
      • via reqwest (blocking only)
    • pass context for scheme specific configuration, like timeouts
  • git://
    • V1 handshake
      • send values + receive data with sidebands
      • support for receiving 'shallow' refs in case the remote repository is shallow itself (I presume)
        • Since V2 doesn't seem to support that, let's skip this until there is an actual need. No completionist :D
    • V2 handshake
      • send command request, receive response with sideband support
  • http(s)://
    • set identity for basic authentication
    • V1 handshake
      • send values + receive data with sidebands
    • V2 handshake
      • send command request, receive response with sideband support
    • 'dumb' - we opt out using this protocol seems too slow to be useful, unless it downloads entire packs for clones?
  • authentication failures are communicated by io::ErrorKind::PermissionDenied, allowing other layers to retry with authentication
  • async support
• server
  • general purpose accept(…) for servers
• API documentation
  • Some examples

Advanced HTTP transport features

• 01 -> async
• 02 -> proxy support
• 03 -> custom request configuration via fn(request)
• 04 -> proxy authentication
• 05 -> reauthentication after redirect

git-protocol

• abstract over protocol versions to allow delegates to deal only with a single way of doing things
• credentials
  • via git-credentials
  • via pure Rust implementation if no git is installed
• handshake
  • parse initial response of V1 and V2 servers
• ls-refs
  • parse V1 refs as provided during handshake
  • parse V2 refs
  • handle empty refs, AKA PKT-LINE(zero-id SP "capabilities^{}" NUL capability-list)
• fetch
  • detailed progress
  • control credentials provider to fill, approve and reject
  • initialize and validate command arguments and features sanely
  • abort early for ls-remote capabilities
  • packfile negotiation
    • delegate can support for all fetch features, including shallow, deepen, etc.
    • receive parsed shallow refs
• push
• API documentation
  • Some examples

git-attributes

• parse git-ignore files (aka git-attributes without the attributes or negation)
• parse git-attributes files
• create an attributes stack, ideally one that includes 'ignored' status from .gitignore files.
  • support for built-in binary macro for -text -diff -merge

git-quote

• ansi-c
  • quote
  • unquote

git-mailmap

• parsing
• lookup and mapping of author names

git-path

• transformations to and from bytes
• conversions between different platforms
• virtual canonicalization for more concise paths via absolutize()
• more flexible canonicalization with symlink resolution for paths which are partially virtual via realpath()
• spec
  • parse
  • check for match

git-pathspec

• parse
• matching of paths

git-refspec

• parse
• matching of references and object names
  • for fetch
  • for push

git-command

• execute commands directly
• execute commands with sh
• support for GIT_EXEC_PATH environment variable with git-sec filter

git-prompt

• open prompts for usernames for example
• secure prompts for password
• use askpass program if available
• signal handling (resetting and restoring terminal settings)
• windows prompts for cmd.exe and mingw terminals

git-note

A mechanism to associate metadata with any object, and keep revisions of it using git itself.

• CRUD for git notes

git-fetchhead

• parse FETCH_HEAD information back entirely
• write typical fetch-head lines

git-discover

• check if a git directory is a git repository
• find a git repository by searching upward
  • define ceilings that should not be surpassed
  • prevent crossing file-systems (non-windows only)
• handle linked worktrees
• a way to handle safe.directory
  • note that it's less critical to support it as gitoxide allows access but prevents untrusted configuration to become effective.

git-date

• parse git dates
• serialize Time

git-credentials

• launch git credentials helpers with a given action
  • built-in git credential program
  • as scripts
  • as absolute paths to programs with optional arguments
  • program name with optional arguments, transformed into git credential-<name>
• helper::main() for easy custom credential helper programs written in Rust

git-filter

Provide base-implementations for dealing with smudge and clean filters as well as filter processes, facilitating their development.

• clean filter base
• smudge filter base
• filter process base

git-sec

Provides a trust model to share across gitoxide crates. It helps configuring how to interact with external processes, among other things.

• integrations
  • git-config
  • git-repository

git-rebase

• obtain rebase status
• drive a rebase operation

git-sequencer

Handle human-aided operations which cannot be completed in one command invocation.

git-lfs

Implement git large file support using the process protocol and make it flexible enough to handle a variety of cases. Make it the best-performing implementation and the most convenient one.

git-glob

• parse pattern
• a type for pattern matching of paths and non-paths, optionally case-insensitively.

git-worktree

• handle the working tree/checkout
  • checkout an index of files, executables and symlinks just as fast as git
    • forbid symlinks in directories
    • handle submodules
    • handle sparse directories
    • handle sparse index
    • linear scaling with multi-threading up to IO saturation
  • supported attributes to affect working tree and index contents
    • eol
    • working-tree-encoding
    • …more
  • filtering
    • text
    • ident
    • filter processes
    • single-invocation clean/smudge filters
• access to all .gitignore/exclude information
• access to all attributes information

git-revision

• describe() (similar to git name-rev)
• parse specifications
  • parsing and navigation
  • revision ranges
  • full date parsing support (depends on git-date)

git-submodule

• CRUD for submodules
• try to handle with all the nifty interactions and be a little more comfortable than what git offers, lay a foundation for smarter git submodules.

git-bitmap

A plumbing crate with shared functionality regarding EWAH compressed bitmaps, as well as other kinds of bitmap implementations.

• EWAH
  • Array type to read and write bits
    • execute closure for each true bit
  • decode on-disk representation
  • encode on-disk representation

git-index

The git staging area.

• read
  • V2 - the default, including long-paths support
  • V3 - extended flags
  • V4 - delta-compression for paths
  • optional threading
    • concurrent loading of index extensions
    • threaded entry reading
  • extensions
    • TREE for speeding up tree generation
    • REUC resolving undo
    • UNTR untracked cache
    • FSMN file system monitor cache V1 and V2
    • EOIE end of index entry
    • IEOT index entry offset table
    • 'link' base indices to take information from, split index
    • 'sdir' sparse directory entries - marker
  • verification of entries and extensions as well as checksum