[Geany-devel] Safe file saving - permissions issue

Fri Apr 2 12:49:02 UTC 2010

As goes from GLib sources, the final replace strategy is backend-dependent.
For local files: (from GLib sources, gio/glocalfileoutputstream.c, handle_overwrite_open):
  /* We use two backup strategies.
   * The first one (which is faster) consist in saving to a
   * tmp file then rename the original file to the backup and the
   * tmp file to the original name. This is fast but doesn't work
   * when the file is a link (hard or symbolic) or when we can't
   * write to the current dir or can't set the permissions on the
   * new file. 
   * The second strategy consist simply in copying the old file
   * to a backup file and rewrite the contents of the file.
   */
For smb:// files (from GVFS sources, daemon/gvfsbackendsmb.c, do_replace)
  /* Backup strategy:
       *
       * By default we:
       *  1) save to a tmp file (that doesn't exist already)
       *  2) rename orig file to backup file
       *     (or delete it if no backup)
       *  3) rename tmp file to orig file
       *
       * However, this can fail if we can't write to the directory.
       * In that case we just truncate the file, after having 
       * copied directly to the backup filename.
       */
SFTP backend strategy is much more complex
So, let's sum it all up

1. There are (at least) 3 different ways to save a file:
  - Use POSIX fopen()/fwrite() API
  - Use g_file_set_contents()
  - Use GIO API

2. On remote filesystems at least first (and probably second) API relies on GIO (via virtual filesystem calls or direct 
GIO calls in newer GLib versions)

3. No API guarantees that data on remote filesystem won't be lost and the save would be "atomic", as the implementation 
is in the backend, which can be buggy etc.

4. The strategy used by GIO on local filesystem seems satisfactory for "safe file saving" needs.

5. All we can do is minimize the probability of data loss due to our code issues. We cannot directly affect troubles in 
GLib and GVFS (IMHO they are messy, but one should contact these projects' teams to improve that)

6. The less (buggy) code there is between Geany and final data transfers (either to disk, or via HTTP/FTP/SMB/SSH/whatever-else), the more reliable our project is

7. That's why the use of direct GIO calls instead of fopen() or g_file_set_contents() is attractive. Consider the chains:
(1) fopen()/fwrite() -> VFS call -> GIO g_file_do_something() -> Backend -> Final data transfer
OR
(2a) g_file_set_contents -> fopen()/fwrite() -> VFS call -> GIO g_file_do_something() -> Backend -> Final data transfer
OR
(2b) g_file_set_contents -> GIO g_file_do_something() -> Backend -> Final data transfer
OR
(3) GIO g_file_do_something() -> Backend -> Final data transfer

8. The only alternative for g_file_replace here is to write our own backup algorithm implementation, preferably based on direct GIO calls.

Fri, 2 Apr 2010 21:35:49 +1100 письмо от Lex Trotman <elextr at gmail.com>:

> On 2 April 2010 03:07, Nick Treleaven <nick.treleaven at btinternet.com> wrote:
> 
> > On Thu, 1 Apr 2010 12:41:13 +0100
> > Nick Treleaven <nick.treleaven at btinternet.com> wrote:
> >
> > > > Btw, the bugs with GVFS didn't appear with GNOME 2.26 (and probably
> > before), so I think we can safely use fopen() and fprintf()/fwrite() there.
> > Then, your proposal about version separation (use GIO with GLib>=2.16 and
> > fopen()/g_file_set_contents() with other) is extremely attractive. Do you
> > know if there is a way to get GLib version at runtime, not at compile-time
> > (so that there is no need to rebuild all the packages for different
> > versions)?
> > > >
> > > > I'm going to start working on it; if this is OK, I'll send a patch
> > ASAP.
> > >
> > > BTW There is another problem that file saving should solve - not
> > > losing existing file data when there is no space to save the new file.
> > > Does g_file_replace_contents() handle this problem? (g_file_set_contents
> > > () does handle this but as you mention has other problems).
> >
> > According to the docs for g_file_replace, which g_file_replace_contents
> > uses internally, this case may be handled:
> >
> > "This will try to replace the file in the safest way possible so that
> > any errors during the writing will not affect an already existing copy
> > of the file. For instance, for local files it may write to a temporary
> > file and then atomically rename over the destination when the stream is
> > closed."
> > http://library.gnome.org/devel/gio/unstable/GFile.html#g-file-replace
> >
> > Hopefully it still keeps permissions if the rename is done(!)
> >
> 
> I would doubt it :-( its creating the new temporary file then renaming that
> to the original name.  This leaves it with the permissions of the temporary
> file ie those of a new file.
> It might work if it copies the attributes of the old file to the temporary
> first, but thats a backend dependent action since permissions depend on the
> filesystem..
> But it does leave the old file unharmed on out of space :-)
> 
> The problem is that remote file systems (most anyway)  do not actually have
> an atomic rename and delete operation.  In fact many actually have race
> conditions when you try to remove the old file and then rename the temporary
> as a non-atomic pair of operations.  So just because it seems to work for
> one application doesn't mean it will allways work for it with a different
> remote file system.  Something like that can be the cause of an empty file.
> 
> Cheers
> Lex
> 
> 
> > Regards,
> > Nick
> > _______________________________________________
> > Geany-devel mailing list
> > Geany-devel at uvena.de
> > http://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
> >
> 
> _______________________________________________
> Geany-devel mailing list
> Geany-devel at uvena.de
> http://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
> 
> 

"Барахолка" на Товары@Mail.Ru. Покупай дешево, продавай выгодно. http://r.mail.ru/cln4270/torg.mail.ru/used/ 