Author Topic: Improved persistence hooks  (Read 1043 times)

JSwigart

  • Newbie
  • *
  • Posts: 16
    • View Profile
Improved persistence hooks
« on: November 27, 2018, 11:23:35 PM »
I think it's fair to say that the current save/load functionality is not necessarily production level. It seems mostly for demonstration purposes, but a real production usage is likely to want to have more control over the save system, such as decoupling the saving of the player/inventory information from the world, and saving/loading sections of the world(in tandem with streaming, etc). So I propose a new save game API. Something like,

Code
void SaveWorldTemplate(BinaryWriter writer, bool onlyModified = true); // saves any world specific information, like the voxel name index map, time of day, etc, so as not to bloat up the individual chunk/column based saves
void SaveWorld(BinaryWriter writer, bool onlyModified = true);
void SaveWorld(BinaryWriter writer,  int chunkX, int chunkZ, bool onlyModified = true); // save the entire vertical column worth of chunks

void LoadWorld(BinaryReader reader); // chunk not provided since it's in the data?

I think this is enough of an API where someone can do incremental spatial based save/loads. These functions should not save player or inventory information.

Related to save/load, I would like to request an OnChunk* event(s) that are sent just before a chunk is unloaded and discarded, so that if the chunk is modified, it may be saved out for later retrieval when that chunk comes back in scope. This would take the place of the never discarding modified chunk functionality.

Edit: I just asked in the channel about how modified chunk persistence is currently treated and currently they are not discarded at all. Perhaps an easy step towards an optional persistence module is providing access to a list of chunks that are out of scope, but still resident, and with that list, a persistence layer can use the above API to save them out and then manually trigger a purge of them(return to pool or whatever). In this way, the persistence module is optional and the default can remain keeping them resident.

Also related to save/load/persistence, an event that can either be triggered via a return value of OnChunkBeforeCreate or a new event like OnChunkRequested so that the chunk may be instructed to wait for a potential asynchronous data load(from disk, from network, etc)

I'm willing to contribute a persistence API if we can come up with a decent way for users to make contributions(github, etc)

Kronnect

  • Administrator
  • Hero Member
  • *****
  • Posts: 6642
    • View Profile
Re: Improved persistence hooks
« Reply #1 on: November 28, 2018, 09:48:16 PM »
Yep. The Load/Save system provides a reference implementation for a more complex system. As with networking, a full fledged subsystem needs to consider way more data that just chunks and voxels hence the current implementation serves basic (but functional) demonstration purposes. For instance, you will want to save any foe position and its state as well.

I'll take a look at your proposal set of functions and add them to VP. Thank you.

JSwigart

  • Newbie
  • *
  • Posts: 16
    • View Profile
Re: Improved persistence hooks
« Reply #2 on: November 30, 2018, 12:08:06 AM »
I think the most significant bit that would have to be supported in the guts of the VP chunk generation in order to have the tools for the different use cases of persistence is the ability to mark a chunk as deferred built or otherwise identify whether or not a chunk is in its expected state. In a game with persistence, the loading of a chunk will likely need to hit the hard drive, network, or a database in order to fetch the data for a chunk. Therefore, the chunk build pipeline either needs to be able to wait on those results, or alternately one could kick off a data fetch in the callback right now and just let it default to the procedural voxel data and when the query comes back basically set the data of the chunk again. That might be preferrable from the perspective that it wouldn't leave holes, but i think there would at least need to be some capability to differentiate between a chunk that is built with default procedural data, and a chunk that has been marked by the user as 'good to go', due to whether or not their data query has come back yet. This flag would probably need to be checked as part of the requirements for displaying the load screen in the minimum chunk radius setting.

Maybe an alternate approach to muddying the chunk build process with async support could be some sort of optional chunk caching module, where the chunk build process won't even be kicked off for a chunk without querying this module that a given chunk x,z is ready to go or something.