Changeset 25528 in project
- Timestamp:
- 11/20/11 19:46:01 (9 years ago)
- Location:
- release/4/ugarit/trunk
- Files:
-
- 1 deleted
- 5 edited
Legend:
- Unmodified
- Added
- Removed
-
release/4/ugarit/trunk/LICENCE.txt
r20271 r25528 1 1 Copyright (c) 2008-2009, Warhead.org.uk Ltd 2 Copyright (c) 2009-2011, Alaric Snell-Pym 2 3 3 4 All rights reserved. -
release/4/ugarit/trunk/README.txt
r25527 r25528 611 611 entirely uploaded. 612 612 613 Actually, there is *one* caveat: blocks that were uploaded, but never 614 make it into a finished snapshot, will be marked as "referenced" but 615 there's no snapshot to delete to un-reference them, so they'll never 616 be removed when you delete snapshots. (Not that snapshot deletion is 617 implemented yet, mind). If this becomes a problem for people, we could 618 write a "garbage collect" tool that regenerates the reference counts 619 in an archive, leading to unused blocks (with a zero refcount) being 620 unlinked. 621 613 622 ## Should I share a single large archive between all my filesystems? 614 623 … … 637 646 the archive at any point, or who can read the archive itself at their 638 647 leisure. 648 649 Ugarit's splitlog backend creates files with "rw-------" permissions 650 out of the box to try and prevent this. This is a pain for people who 651 want to share archives between UIDs, but we can add a configuration 652 option to override this if that becomes a problem. 639 653 640 654 ### Reading your data … … 846 860 stats in the index of how many byte ranges are unused, and how many 847 861 bytes unused, in each file, and report them in the info admin 848 interface, along with the option to compact any or all files. 862 interface, along with the option to compact any or all files. We'll 863 need to store refcounts in the backend metadata (should we log 864 reuses, then, so the metadata can always be reconstructed, or just 865 set them to NULL on a reconstruct); when this is enabled on an 866 existing archive with no refcounts, default them to NULL, and treat 867 a NULL refcount as "infinity". 849 868 850 869 * Have read-only and unlinkable and block size config flags in the … … 888 907 this, we might need to log an "unknown" for that block key / archive 889 908 pair, rather than assuming the block is not there, and check it 890 later. Users 891 will be given an admin command to notify the backend of an archive 892 going missing forever, which will cause it to be removed from the 893 cache. Affected blocks should be examined and re-replicated if their 894 replication count is now too low. Another command should be 909 later. Users will be given an admin command to notify the backend of 910 an archive going missing forever, which will cause it to be removed 911 from the cache. Affected blocks should be examined and re-replicated 912 if their replication count is now too low. Another command should be 895 913 available to warn of impending deliberate removal, which will again 896 914 remove the archive from the cluster and re-replicate, the difference 897 915 being that the disappearing archive is usable for re-replicating 898 916 FROM, so this is a safe operation for blocks that are only on that 899 one archive. The individual physical archives 900 that we put replication on top of won't be "valid" archives unless 901 they are 100% replicated, as they'll contain references to blocks 902 that are on other archives. It might be a good idea to mark them as 903 such with a special tag to avoid people trying to restore directly 904 from them. A copy of the replication configuration could be stored 905 under a special tag to mark this fact, and to enable easy finding of 906 the proper replicated archive to work from. There should be a 907 configurable option to snapshot the cache to the archives whenever 908 the replicated archive is closed, too. The command line to the 909 backend, "backend-replicated", should point to an sqlite file for 910 the configuration and cache, and users should use admin commands to 911 add/remove/modify archives in the cluster. 917 one archive. The individual physical archives that we put 918 replication on top of won't be "valid" archives unless they are 100% 919 replicated, as they'll contain references to blocks that are on 920 other archives. It might be a good idea to mark them as such with a 921 special tag to avoid people trying to restore directly from them; 922 the frontend should complain if you attempt to directly use an 923 archive with the special tag in place. A copy of the replication 924 configuration could be stored under a special tag to mark this fact, 925 and to enable easy finding of the proper replicated archive to work 926 from. There should be a configurable option to snapshot the cache to 927 the archives whenever the replicated archive is closed, too. The 928 command line to the backend, "backend-replicated", should point to 929 an sqlite file for the configuration and cache, and users should use 930 admin commands to add/remove/modify archives in the cluster. 912 931 913 932 ## Core … … 917 936 flag and, if set, display what path we're working on, and perhaps a 918 937 quick summary of the bytes/blocks stored/skipped stats. 919 920 * Look at http://bugs.call-cc.org/ticket/492 - can we now ditch our921 own POSIX wrappers and use this egg?922 938 923 939 * Log all WARNINGs produced during a snapshot job, and attach them to … … 985 1001 given tag. Or just remove X out of Y snapshots older than a given 986 1002 number of days on a given tag. We have the core support for this; 987 just find a snapshot and `unlink-directory!` it , leaving a dangling988 pointer from the snapshot, and write the snapshot handling code to989 expect this. Again, check Box Backup for that.1003 just find a snapshot and `unlink-directory!` its contents, leaving a 1004 dangling pointer from the snapshot, and write the snapshot handling 1005 code to expect this. Again, check Box Backup for that. 990 1006 991 1007 * Option, when backing up, to not cross mountpoints … … 1032 1048 any required top-level snapshot objects if the dump doesn't start 1033 1049 from a snapshot) and putting it onto a specified tag. The 1034 inten sion is that this format can be used to migrate your stuff1050 intention is that this format can be used to migrate your stuff 1035 1051 between archives, perhaps to change to a better backend. 1036 1052 … … 1038 1054 1039 1055 * Better error messages 1040 1041 * Line editing in the "explore" CLI, ideally with tab completion1042 1056 1043 1057 * API mode: Works something like the backend API, except at the … … 1050 1064 archive, rather than needing to use explore mode. Also the option to 1051 1065 extract given just a block key (useful when reading from keys logged 1052 manually at snapshot time , or from a backend that has a tag log).1066 manually at snapshot time). 1053 1067 1054 1068 * FUSE/9p support. Mount it as a read-only filesystem :-D Then … … 1156 1170 every write, in order to improve performance. Counting blocks and 1157 1171 bytes uploaded / reused, and file cache bytes as well as hits; 1158 reporting same in snapshot UI and logging same to snapshot metadata. 1172 reporting same in snapshot UI and logging same to snapshot 1173 metadata. Switched to the `posix-extras` egg and ditched our own 1174 `posixextras.scm` wrappers. Used the `parley` egg in the `ugarit 1175 explore` CLI for line editing. 1159 1176 1160 1177 * 1.0.1: Consistency check on read blocks by default. Removed warning -
release/4/ugarit/trunk/ugarit-core.scm
r25527 r25528 100 100 (use miscmacros) 101 101 (use posix) 102 (use posix-extras) 102 103 (use crypto-tools) 103 104 (use stty) … … 140 141 (setter file-cache-bytes) ; count of file cache bytes saved 141 142 ) 142 143 (include "posixextras.scm")144 143 145 144 (define file-cache-commit-interval 1000) -
release/4/ugarit/trunk/ugarit.meta
r25479 r25528 3 3 (license "BSD") 4 4 (category data) 5 (needs miscmacros sql-de-lite crypto-tools srfi-37 stty matchable autoload regex tiger-hash )5 (needs miscmacros sql-de-lite crypto-tools srfi-37 stty matchable autoload regex tiger-hash posix-extras parley) 6 6 (optional lzma z3 tiger-hash sha2 aes) 7 7 (author "Alaric Snell-Pym") -
release/4/ugarit/trunk/ugarit.scm
r25527 r25528 6 6 (use matchable) 7 7 (use regex) 8 (use parley) 8 9 9 10 (define (bit? i b) … … 134 135 ;; To get started, call with '() as directory-key and path 135 136 (define (explore-archive archive directory-key path quit-continuation) 136 (printf "~A> " (apply string-append (map (lambda (element) (string-append "/" element)) path))) 137 (let ((line (read-line))) 137 (let ((line (parley (sprintf "~A> " (apply string-append (map (lambda (element) (string-append "/" element)) path)))))) 138 138 (if (eq? line #!eof) 139 139 (quit-continuation (void)))
Note: See TracChangeset
for help on using the changeset viewer.