History

Get closer to implementing a simple TREE extension decoding (#293)

Sebastian Thiel committed 4 years ago

Commit 49fcb6f1 parent 07e8fb27 files · +77 −13Browse files

Unified · Split

7 files
  1. Cargo.lock +1 −0
  2. README.md +12 −11
  3. git-index/Cargo.toml +1 −0
  4. git-index/src/decode.rs +17 −1
  5. git-index/src/extension/mod.rs +20 −0
  6. git-index/src/extension/tree.rs +25 −0
  7. git-index/tests/file/mod.rs +1 −1
Cargo.lock+1−0
@@ -1234,6 +1234,7 @@ dependencies = [ · 1233 unmodified lines
1234 "memmap2",1234 "memmap2",
1235 "quick-error",1235 "quick-error",
1236 "serde",1236 "serde",
1237 "smallvec",
1237]1238]
1238 1239
1239[[package]]1240[[package]]
README.md+12−11
@@ -243,17 +243,6 @@ Provide a CLI to for the most basic user journey: · 242 unmodified lines
243 as well. What would an MVP look like? Maybe even something that could ship with gitoxide.243 as well. What would an MVP look like? Maybe even something that could ship with gitoxide.
244* [ ] A truly awesome history rewriter which makes it easy to understand what happened while avoiding all pitfalls. Think BFG, but more awesome, if that's possible.244* [ ] A truly awesome history rewriter which makes it easy to understand what happened while avoiding all pitfalls. Think BFG, but more awesome, if that's possible.
245* [ ] `git-tui` should learn a lot from [fossil-scm] regarding the presentation of data. Maybe [this](https://github.com/Lutetium-Vanadium/requestty/) can be used for prompts. Probably [magit] has a lot to offer, too.245* [ ] `git-tui` should learn a lot from [fossil-scm] regarding the presentation of data. Maybe [this](https://github.com/Lutetium-Vanadium/requestty/) can be used for prompts. Probably [magit] has a lot to offer, too.
246* An event-based database that uses commit messages to store deltas, while occasionally aggregating the actual state in a tree. Of course it's distributed by nature, allowing
247 people to work offline.
248 - It's abstracted to completely hide the actual data model behind it, allowing for all kinds of things to be implemented on top.
249 - Commits probably need a nanosecond component for the timestamp, which can be added via custom header field.
250 - having recording all changes allows for perfect merging, both on the client or on the server, while keeping a natural audit log which makes it useful for mission critical
251 databases in business.
252 * Can markdown be used as database so issue-trackers along with meta-data could just be markdown files which are mostly human-editable? Could user interfaces
253 be meta-data aware and just hide the meta-data chunks which are now editable in the GUI itself? Doing this would make conflicts easier to resolve than an `sqlite`
254 database.
255 * ~~A git-backend for `sqlite` which should allow embedding sqlite databases into git repositories, which in turn can be used for bug-trackers, wikis or other
256 features, making for a fully distributed github like experience, maybe.~~
257 246
258### Ideas for Spin-Offs247### Ideas for Spin-Offs
259 248
@@ -265,6 +254,18 @@ Provide a CLI to for the most basic user journey: · 5 unmodified lines
265* [ ] A [syncthing] like client/server application. This is to demonstrate how lower-level crates can be combined into custom applications that use254* [ ] A [syncthing] like client/server application. This is to demonstrate how lower-level crates can be combined into custom applications that use
266 only part of git's technology to achieve their very own thing. Watch out for big file support, multi-device cross-syncing, the possibility for255 only part of git's technology to achieve their very own thing. Watch out for big file support, multi-device cross-syncing, the possibility for
267 untrusted destinations using full-encryption, case-insensitive and sensitive filesystems, and extended file attributes as well as ignore files.256 untrusted destinations using full-encryption, case-insensitive and sensitive filesystems, and extended file attributes as well as ignore files.
257* An event-based database that uses commit messages to store deltas, while occasionally aggregating the actual state in a tree. Of course it's distributed by nature, allowing
258 people to work offline.
259 - It's abstracted to completely hide the actual data model behind it, allowing for all kinds of things to be implemented on top.
260 - Commits probably need a nanosecond component for the timestamp, which can be added via custom header field.
261 - having recording all changes allows for perfect merging, both on the client or on the server, while keeping a natural audit log which makes it useful for mission critical
262 databases in business.
263 * **Applications**
264 - Can markdown be used as database so issue-trackers along with meta-data could just be markdown files which are mostly human-editable? Could user interfaces
265 be meta-data aware and just hide the meta-data chunks which are now editable in the GUI itself? Doing this would make conflicts easier to resolve than an `sqlite`
266 database.
267 - A time tracker - simple data, very likely naturally conflict free, and interesting to see it in terms of teams or companies using it with maybe GitHub as Backing for authentication.
268 - How about supporting multiple different trackers, as in different remotes?
268 269
269[syncthing]: https://github.com/syncthing/syncthing270[syncthing]: https://github.com/syncthing/syncthing
270[fossil-scm]: https://www.fossil-scm.org271[fossil-scm]: https://www.fossil-scm.org
git-index/Cargo.toml+1−0
@@ -24,6 +24,7 @@ memmap2 = "0.5.0" · 23 unmodified lines
24filetime = "0.2.15"24filetime = "0.2.15"
25 25
26serde = { version = "1.0.114", optional = true, default-features = false, features = ["derive"] }26serde = { version = "1.0.114", optional = true, default-features = false, features = ["derive"] }
27smallvec = "1.7.0"
27 28
28[dev-dependencies]29[dev-dependencies]
29git-testtools = { path = "../tests/tools"}30git-testtools = { path = "../tests/tools"}
git-index/src/decode.rs+17−1
@@ -67,13 +67,29 @@ mod error { · 66 unmodified lines
67 }67 }
68 }68 }
69}69}
70
71pub use error::Error;70pub use error::Error;
72 71
73impl State {72impl State {
74 pub fn from_bytes(data: &[u8], timestamp: FileTime, object_hash: git_hash::Kind) -> Result<Self, Error> {73 pub fn from_bytes(data: &[u8], timestamp: FileTime, object_hash: git_hash::Kind) -> Result<Self, Error> {
75 let (version, num_entries, post_header_data) = header::decode(&data, object_hash)?;74 let (version, num_entries, post_header_data) = header::decode(&data, object_hash)?;
76 let start_of_extensions = extension::end_of_index_entry::decode(&data, object_hash);75 let start_of_extensions = extension::end_of_index_entry::decode(&data, object_hash);
76 match start_of_extensions {
77 Some(offset) => {
78 let extensions = extension::Iter::new_without_checksum(&data[offset..], object_hash);
79 for (signature, ext_data) in extensions {
80 match signature {
81 extension::tree::SIGNATURE => {
82 let tree = extension::tree::decode(ext_data, object_hash);
83 todo!("put tree somewhere")
84 }
85 extension::end_of_index_entry::SIGNATURE => {} // skip already done
86 _unknown => {} // skip unknown extensions, too
87 }
88 }
89 todo!("load all extensions in thread, then get IEOT, then possibly multi-threaded entry parsing")
90 }
91 None => todo!("load entries singlge-threaded, then extensions"),
92 }
77 93
78 Ok(State { timestamp, version })94 Ok(State { timestamp, version })
79 }95 }
git-index/src/extension/mod.rsrenamed from git-index/src/extension.rs+20−0
@@ -1,4 +1,5 @@
1use crate::{util::read_u32, Version};1use crate::{util::read_u32, Version};
2use smallvec::SmallVec;
2 3
3const MIN_SIZE: usize = 4 /* signature */ + 4 /* size */;4const MIN_SIZE: usize = 4 /* signature */ + 4 /* size */;
4 5
@@ -10,6 +11,15 @@ fn decode_header(data: &[u8]) -> (Signature, u32, &[u8]) { · 5 unmodified lines
10 (signature.try_into().unwrap(), read_u32(size), data)11 (signature.try_into().unwrap(), read_u32(size), data)
11}12}
12 13
14pub struct Tree {
15 /// Only set if there are any entries in the index we are associated with.
16 id: Option<tree::NodeId>,
17 name: SmallVec<[u8; 23]>,
18 children: Vec<Tree>,
19}
20
21pub(crate) mod tree;
22
13pub(crate) mod end_of_index_entry {23pub(crate) mod end_of_index_entry {
14 use crate::{decode::header, extension, extension::Signature, util::read_u32};24 use crate::{decode::header, extension, extension::Signature, util::read_u32};
15 25
@@ -69,6 +79,16 @@ mod iter { · 53 unmodified lines
69 data: data_at_beginning_of_extensions_and_truncated,79 data: data_at_beginning_of_extensions_and_truncated,
70 }80 }
71 }81 }
82
83 pub fn new_without_checksum(data_at_beginning_of_extensions: &'a [u8], object_hash: git_hash::Kind) -> Self {
84 let end = data_at_beginning_of_extensions
85 .len()
86 .checked_sub(object_hash.len_in_bytes())
87 .expect("someone asserted that there is at least one extension");
88 Iter {
89 data: &data_at_beginning_of_extensions[..end],
90 }
91 }
72 }92 }
73 93
74 impl<'a> Iterator for Iter<'a> {94 impl<'a> Iterator for Iter<'a> {
git-index/src/extension/tree.rs+25−0
@@ -0,0 +1,25 @@
1use crate::extension::{Signature, Tree};
2
3pub const SIGNATURE: Signature = *b"TREE";
4
5pub struct NodeId {
6 /// The id of the directory tree of the associated tree object.
7 id: git_hash::ObjectId,
8 /// The amount of non-tree entries contained within, and definitely not zero.
9 entry_count: u32,
10}
11
12/// A recursive data structure
13pub fn decode(data: &[u8], object_hash: git_hash::Kind) -> Option<Tree> {
14 todo!("decode tree")
15}
16
17#[cfg(test)]
18mod tests {
19 use super::*;
20
21 #[test]
22 fn size_of_tree() {
23 assert_eq!(std::mem::size_of::<Tree>(), 88);
24 }
25}
git-index/tests/file/mod.rs+1−1
@@ -6,7 +6,7 @@ mod init { · 5 unmodified lines
6 }6 }
7 7
8 #[test]8 #[test]
9 fn read_v2() {9 fn read_v2_with_single_entry_tree() {
10 let file = file("v2");10 let file = file("v2");
11 assert_eq!(file.version(), Version::V2);11 assert_eq!(file.version(), Version::V2);
12 }12 }

Keyboard shortcuts

General

Show keyboard shortcuts
?
Focus the current search
/

Issues and pull requests

Open filters
f

Issues, pull requests and Inbox

Select next item
j↓
Select previous item
k↑
Open selected item
Enter
Dismiss filters or clear selection
Esc

Repository code

Go to file
t