Changeset 25525 in project for release/4/ugarit/trunk/README.txt


Ignore:
Timestamp:
11/19/11 18:30:21 (10 years ago)
Author:
Alaric Snell-Pym
Message:

ugarit: Only commit the file-cache sqlite database periodically, to improve performance on lots of small files.

File:
1 edited

Legend:

Unmodified
Added
Removed
  • release/4/ugarit/trunk/README.txt

    r25521 r25525  
    820820  operations performed without a subsequent `flush` might not "stick" in
    821821  failure cases (make `close!` have an implicit `flush`, of
    822   course). Then use this to let splitlog and cache backends buffer
    823   sqlite `INSERT`s and then spit them out in a single transaction per
    824   `flush`/`close` or when the buffer hits a determined size limit, to
    825   improve throughput.
     822  course). Use this to force an immediate commit in backends that use
     823  sqlite, as well as the current practice of committing at close, tag
     824  operations, and whenever it seems like a while has passed since the
     825  last time. Make the frontend flush at crucial points.
    826826
    827827* Implement "info" admin commands for all backends, that list any
     
    848848  metadata file, settable via admin commands.
    849849
    850 * Optional support in backends for keeping a log of tag changes, and
    851   admin commands to read the log.
     850* For people doing remote backups who want to not hog resources, write
     851  a proxy backend that throttles bandwidth usage. Make it record the
     852  time it last sent a request to the backend, and the number of bytes
     853  read and written; then when a new request comes in, delay it until
     854  at least the largest of (write bandwidth quota * bytes written) and
     855  (read bandwidth quota * bytes read) seconds has passed since the
     856  last request was sent. NOTE: Start the clock when SENDING, so the
     857  time spent handling the request is already counting towards
     858  bandwidth quotas, or it won't be fair.
    852859
    853860* Support for SFTP as a storage backend. Store one file per block, as
     
    855862  http://tools.ietf.org/html/draft-ietf-secsh-filexfer-13 for sftp
    856863  protocol specs; popen an `ssh -p sftp` connection to the server then
    857   talk that simple binary protocol. Tada!
    858 
    859 * Support for S3 as a storage backend. There is now an S3 egg!
     864  talk that simple binary protocol. Tada! Ideally make an sftp egg,
     865  then a "ugarit-backend-sftp" egg to keep the dependencies optional.
     866
     867* Support for S3 as a storage backend. There is now an S3 egg! Make an
     868  "ugarit-backend-s3" egg to keep the dependencies optional.
    860869
    861870* Support for replicated archives. This will involve a special storage
     
    908917  human-supplied tag names match a regular expression. Leave space for
    909918  system-only tag names for storing archive metadata; suggest making a
    910   # sign illegal in tag names.
     919  hash sign illegal in tag names.
    911920
    912921* Clarify what characters are legal in block keys. Ugarit will only
     
    10411050  manually at snapshot time, or from a backend that has a tag log).
    10421051
    1043 * FUSE support. Mount it as a read-only filesystem :-D Then consider
    1044   adding Fossil-style writing to the `current` of a snapshot, with
    1045   copy-on-write of blocks to a buffer area on the local disk, then the
    1046   option to make a snapshot of `current`.
     1052* FUSE/9p support. Mount it as a read-only filesystem :-D Then
     1053  consider adding Fossil-style writing to the `current` of a snapshot,
     1054  with copy-on-write of blocks to a buffer area on the local disk,
     1055  then the option to make a snapshot of `current`. Put these into
     1056  separate "ugarit-frontend-9p" and "ugarit-frontend-fuse" eggs, to
     1057  control the dependencies.
    10471058
    10481059* Filesystem watching. Even with the hash-caching trick, a snapshot
     
    11401151# Version history
    11411152
     1153* 1.0.2: Made the file cache also commit periodically, rather than on
     1154  every write, in order to improve performance.
     1155
    11421156* 1.0.1: Consistency check on read blocks by default. Removed warning
    11431157  about deletions from backend-cache; we need a new mechanism to
Note: See TracChangeset for help on using the changeset viewer.