dMZX Forums: Add VFS -> MegaZeux Feature Requests -> Tracker

This feature will add stdio abstraction functions across MZX to allow for file caching, virtual files, ZIP mounting, and/or other potentially useful things. It needs to be implemented in several parts:

Preparation

• Add stdio abstraction interface IMPLEMENTED (2.92b)
  
• Add Windows UTF-8 layer to stdio abstraction IMPLEMENTED (2.92c)
  
• Refactor directory functions out of util.c, add Windows UTF-8 layer IMPLEMENTED (2.92c)
  
• Add stdio abstraction replacement for ftell_and_rewind IMPLEMENTED (2.92d)
  
• Add stdio abstraction implementation for fgets, memfile.h version mfgets from memsafegets IMPLEMENTED (2.92d) (GIT 46032111)
  
• Move memsafegets from util.c to fsafeopen.c or memfile.h IMPLEMENTED (2.92d) (GIT 46032111)
  
• Move vfs.c -> vfile.c, zip.c, dir.c, fsafeopen.c, memfile.h, zip/ into new folder io/ with Makefile fragment IMPLEMENTED (2.92d) (GIT 8fe56e0f)
  
• Refactor path functions out of util.c and move to io/, make generally more consistent/sane/maybe have versions that don't require duplicating the string IMPLEMENTED (2.92d) (GIT a03f078d)
  
• Replace mem_* functions with memfile IMPLEMENTED (2.92d) (GIT c0bda793)

Implementation

• Replace stdio/misc. unistd functions with stdio abstraction across MegaZeux. IMPLEMENTED (2.93) (fixes the stdio portion of #718)
  
  • World file IO IMPLEMENTED (GIT ef777e7c)
    
  • main.c, config file, file manager, GLSL shaders, util.c, and other core misc. IMPLEMENTED (GIT a6bc67a9)
    
  • Charsets/palettes/programs/input file/output file (fsafeopen) IMPLEMENTED (GIT 649377d4)
    
  • helpsys IMPLEMENTED (GIT 649377d4)
    
  • Screenshots IMPLEMENTED (GIT e8bb8dd6)
    
  • Audio misc. IMPLEMENTED (various patches)
    
  • Editor misc. IMPLEMENTED (GIT 5353a43d)
    
  • Network misc. IMPLEMENTED (GIT 5353a43d)
    
  • Utils misc. IMPLEMENTED (GIT 5353a43d)
  
  
• Add VFS. IMPLEMENTED (GIT 83755a5c)
  
• Add VFS file caching support functions.
  
• Integrate VFS into vio, add vio virtual file flags.
  
• Integrate VFS file caching into vio, add vio file caching flags. (fixes the slow file access portion of #778)
  
• VFS configuration options, use virtual files to potentially speed up configure unit test and test world runner.

Stretch Goals

• Rewrite legacy world decryption to use a temporary vfile instead of rewriting the file for platforms with adequate memory. (would fix #776) IMPLEMENTED (GIT 1d5202cf)
  
• Potentially add aggressive precaching of files in the current directory for the 3DS port (would improve the above fix for #778)
  
• Transparent ZIP file access and/or ZIP mounting (would help simplify Emscripten frontend archive code, potentially help work around #789)
  
• Potentially make fsafetranslate handle DOS name translation in unambiguous situations where the DOS name is specified in the MZX file and the extended name file exists in the archive. (could partially or entirely fix numerous games)

Other issues that may be partially or entirely resolved by this:

• The PS Vita port has slow load issues due to what seems to be a bug with setvbuf. VFS caching would bypass this and obsolete current hacks in that branch.

This post has been edited by Lachesis: 26 January 2022 - 12:07 PM

Most of MZX's core code (aside from anything related to fsafeopen.c or graphics.c) is using the vio functions now.

The VFS data structure, internal functions, and unit tests are implemented as of GIT 83755a5c.