Changeset 25528 in project


Ignore:
Timestamp:
11/20/11 19:46:01 (10 years ago)
Author:
Alaric Snell-Pym
Message:

ugarit: Used the parley egg for command-line editing.

Location:
release/4/ugarit/trunk
Files:
1 deleted
5 edited

Legend:

Unmodified
Added
Removed
  • release/4/ugarit/trunk/LICENCE.txt

    r20271 r25528  
    11Copyright (c) 2008-2009, Warhead.org.uk Ltd
     2Copyright (c) 2009-2011, Alaric Snell-Pym
    23
    34All rights reserved.
  • release/4/ugarit/trunk/README.txt

    r25527 r25528  
    611611entirely uploaded.
    612612
     613Actually, there is *one* caveat: blocks that were uploaded, but never
     614make it into a finished snapshot, will be marked as "referenced" but
     615there's no snapshot to delete to un-reference them, so they'll never
     616be removed when you delete snapshots. (Not that snapshot deletion is
     617implemented yet, mind). If this becomes a problem for people, we could
     618write a "garbage collect" tool that regenerates the reference counts
     619in an archive, leading to unused blocks (with a zero refcount) being
     620unlinked.
     621
    613622## Should I share a single large archive between all my filesystems?
    614623
     
    637646the archive at any point, or who can read the archive itself at their
    638647leisure.
     648
     649Ugarit's splitlog backend creates files with "rw-------" permissions
     650out of the box to try and prevent this. This is a pain for people who
     651want to share archives between UIDs, but we can add a configuration
     652option to override this if that becomes a problem.
    639653
    640654### Reading your data
     
    846860  stats in the index of how many byte ranges are unused, and how many
    847861  bytes unused, in each file, and report them in the info admin
    848   interface, along with the option to compact any or all files.
     862  interface, along with the option to compact any or all files. We'll
     863  need to store refcounts in the backend metadata (should we log
     864  reuses, then, so the metadata can always be reconstructed, or just
     865  set them to NULL on a reconstruct); when this is enabled on an
     866  existing archive with no refcounts, default them to NULL, and treat
     867  a NULL refcount as "infinity".
    849868
    850869* Have read-only and unlinkable and block size config flags in the
     
    888907  this, we might need to log an "unknown" for that block key / archive
    889908  pair, rather than assuming the block is not there, and check it
    890   later. Users
    891   will be given an admin command to notify the backend of an archive
    892   going missing forever, which will cause it to be removed from the
    893   cache. Affected blocks should be examined and re-replicated if their
    894   replication count is now too low. Another command should be
     909  later. Users will be given an admin command to notify the backend of
     910  an archive going missing forever, which will cause it to be removed
     911  from the cache. Affected blocks should be examined and re-replicated
     912  if their replication count is now too low. Another command should be
    895913  available to warn of impending deliberate removal, which will again
    896914  remove the archive from the cluster and re-replicate, the difference
    897915  being that the disappearing archive is usable for re-replicating
    898916  FROM, so this is a safe operation for blocks that are only on that
    899   one archive. The individual physical archives
    900   that we put replication on top of won't be "valid" archives unless
    901   they are 100% replicated, as they'll contain references to blocks
    902   that are on other archives. It might be a good idea to mark them as
    903   such with a special tag to avoid people trying to restore directly
    904   from them. A copy of the replication configuration could be stored
    905   under a special tag to mark this fact, and to enable easy finding of
    906   the proper replicated archive to work from. There should be a
    907   configurable option to snapshot the cache to the archives whenever
    908   the replicated archive is closed, too. The command line to the
    909   backend, "backend-replicated", should point to an sqlite file for
    910   the configuration and cache, and users should use admin commands to
    911   add/remove/modify archives in the cluster.
     917  one archive. The individual physical archives that we put
     918  replication on top of won't be "valid" archives unless they are 100%
     919  replicated, as they'll contain references to blocks that are on
     920  other archives. It might be a good idea to mark them as such with a
     921  special tag to avoid people trying to restore directly from them;
     922  the frontend should complain if you attempt to directly use an
     923  archive with the special tag in place. A copy of the replication
     924  configuration could be stored under a special tag to mark this fact,
     925  and to enable easy finding of the proper replicated archive to work
     926  from. There should be a configurable option to snapshot the cache to
     927  the archives whenever the replicated archive is closed, too. The
     928  command line to the backend, "backend-replicated", should point to
     929  an sqlite file for the configuration and cache, and users should use
     930  admin commands to add/remove/modify archives in the cluster.
    912931
    913932## Core
     
    917936  flag and, if set, display what path we're working on, and perhaps a
    918937  quick summary of the bytes/blocks stored/skipped stats.
    919 
    920 * Look at http://bugs.call-cc.org/ticket/492 - can we now ditch our
    921   own POSIX wrappers and use this egg?
    922938
    923939* Log all WARNINGs produced during a snapshot job, and attach them to
     
    9851001  given tag. Or just remove X out of Y snapshots older than a given
    9861002  number of days on a given tag. We have the core support for this;
    987   just find a snapshot and `unlink-directory!` it, leaving a dangling
    988   pointer from the snapshot, and write the snapshot handling code to
    989   expect this. Again, check Box Backup for that.
     1003  just find a snapshot and `unlink-directory!` its contents, leaving a
     1004  dangling pointer from the snapshot, and write the snapshot handling
     1005  code to expect this. Again, check Box Backup for that.
    9901006
    9911007* Option, when backing up, to not cross mountpoints
     
    10321048  any required top-level snapshot objects if the dump doesn't start
    10331049  from a snapshot) and putting it onto a specified tag. The
    1034   intension is that this format can be used to migrate your stuff
     1050  intention is that this format can be used to migrate your stuff
    10351051  between archives, perhaps to change to a better backend.
    10361052
     
    10381054
    10391055* Better error messages
    1040 
    1041 * Line editing in the "explore" CLI, ideally with tab completion
    10421056
    10431057* API mode: Works something like the backend API, except at the
     
    10501064  archive, rather than needing to use explore mode. Also the option to
    10511065  extract given just a block key (useful when reading from keys logged
    1052   manually at snapshot time, or from a backend that has a tag log).
     1066  manually at snapshot time).
    10531067
    10541068* FUSE/9p support. Mount it as a read-only filesystem :-D Then
     
    11561170  every write, in order to improve performance. Counting blocks and
    11571171  bytes uploaded / reused, and file cache bytes as well as hits;
    1158   reporting same in snapshot UI and logging same to snapshot metadata.
     1172  reporting same in snapshot UI and logging same to snapshot
     1173  metadata. Switched to the `posix-extras` egg and ditched our own
     1174  `posixextras.scm` wrappers. Used the `parley` egg in the `ugarit
     1175  explore` CLI for line editing.
    11591176
    11601177* 1.0.1: Consistency check on read blocks by default. Removed warning
  • release/4/ugarit/trunk/ugarit-core.scm

    r25527 r25528  
    100100(use miscmacros)
    101101(use posix)
     102(use posix-extras)
    102103(use crypto-tools)
    103104(use stty)
     
    140141  (setter file-cache-bytes)                 ; count of file cache bytes saved
    141142  )
    142 
    143 (include "posixextras.scm")
    144143
    145144(define file-cache-commit-interval 1000)
  • release/4/ugarit/trunk/ugarit.meta

    r25479 r25528  
    33 (license "BSD")
    44 (category data)
    5  (needs miscmacros sql-de-lite crypto-tools srfi-37 stty matchable autoload regex tiger-hash)
     5 (needs miscmacros sql-de-lite crypto-tools srfi-37 stty matchable autoload regex tiger-hash posix-extras parley)
    66 (optional lzma z3 tiger-hash sha2 aes)
    77 (author "Alaric Snell-Pym")
  • release/4/ugarit/trunk/ugarit.scm

    r25527 r25528  
    66(use matchable)
    77(use regex)
     8(use parley)
    89
    910(define (bit? i b)
     
    134135;; To get started, call with '() as directory-key and path
    135136(define (explore-archive archive directory-key path quit-continuation)
    136    (printf "~A> " (apply string-append (map (lambda (element) (string-append "/" element)) path)))
    137    (let ((line (read-line)))
     137   (let ((line (parley (sprintf "~A> " (apply string-append (map (lambda (element) (string-append "/" element)) path))))))
    138138      (if (eq? line #!eof)
    139139         (quit-continuation (void)))
Note: See TracChangeset for help on using the changeset viewer.