Fixing a Custom Plugin To Prevent Data Loss on Server Crash

Status
This thread has been locked.
InMud7

InMud7

Right where I'm supposed to be ;)
Supreme
Feedback score
1
Posts
154
Reactions
130
Resources
3
I STRONGLY recommend reading this ENTIRE post rather than just skipping to the end and assuming you can do it. If you contact me and it is obvious you did not read it, your post will be ignored.

I have hired multiple developers to work on this plugin over time. It allows players to store items in an infinite storage using an enderchest looking block to access it. It used to use MySQL but there was an issue that caused players to only be able to view their infinite storage at random times and like half the time it wouldn't work. The developer I worked with spent a solid 2 weeks - 1 month or so working on trying to solve this and was unable to. So I eventually switched to flatfile storage. An example file looks like:
Code: 
name: Mochi3Rabbit
sort: NORMAL
items: |
  rO0ABXcEAAAAy3NyABpvcmcuYnVra2l0LnV0aWwuaW8uV3JhcHBlcvJQR+zxEm8FAgABTAADbWFw
  dAAPTGphdmEvdXRpbC9NYXA7eHBzcgA1Y29tLmdvb2dsZS5jb21tb24uY29sbGVjdC5JbW11dGFi

This has worked fine up until the day of the server crash. When the server crashed many players lost access to their infinite storages (both their stored locations and actual items stored in it). When I checked the server files I noticed 3 files had normal data stored and 7 player files had their data wiped. The only thing that remained was:
Code: 
name: DOMGODFTW
sort: NORMAL

I'm hiring someone to fix this issue so player's data isn't lost when the server crashes (or only the last few min is at worst) without impacting server performance. Also, any bugs my friend and I find with the plugin after you work on it should also be fixed. Further discussion of the issue can be found here.

Edit: Ghast below suggested "save the data on every enderchest close event. Not only is this the only most efficient one, but if you put it into parallel streams it should be even better."
Edit 2: YoloSanta also suggested some code to add. Please take a look at it below.

If you are interested in this job, DM me with the following info:
1. Your portfolio
2. A link to this page
3. How you think the issue could be solved / prevented, or if you agree with Ghast's & YoloSanta's suggestions. Please note that the server was changed from MySQL to Flatfile because of the recurring issues with MySQL.

Payment: $15
Payment Terms: Payment will be sent after I have fully bug tested it on my own server and all bugs have been fixed. As some assurance to you, after I have sent the full payment, please send me the source code (including maven file).
Time Frame: 3 Days

Final Notes:
- Plugin should work for Paper / Spigot 1.14.x+

Not sure if you can update the code? Want to see what the code looks like? Here's an example:
 
Last edited:
Click to expand...
PebbleHost
High performance, consistent uptime and fast support. Minecraft hosting that just works.
Ghast

Ghast

Founding Father of Hypocrisy - https://artemis.ac
Supreme
Feedback score
54
Posts
2,096
Reactions
3,285
Resources
79
PiggiesGoSqueal said:
I STRONGLY recommend reading this ENTIRE post rather than just skipping to the end and assuming you can do it. If you contact me and it is obvious you did not read it, your post will be ignored.

I have hired multiple developers to work on this plugin over time. It allows players to store items in an infinite storage using an enderchest looking block to access it. It used to use MySQL but there was an issue that caused players to only be able to view their infinite storage at random times and like half the time it wouldn't work. The developer I worked with spent a solid 2 weeks - 1 month or so working on trying to solve this and was unable to. So I eventually switched to flatfile storage. An example file looks like:
Code: 
name: Mochi3Rabbit
sort: NORMAL
items: |
  rO0ABXcEAAAAy3NyABpvcmcuYnVra2l0LnV0aWwuaW8uV3JhcHBlcvJQR+zxEm8FAgABTAADbWFw
  dAAPTGphdmEvdXRpbC9NYXA7eHBzcgA1Y29tLmdvb2dsZS5jb21tb24uY29sbGVjdC5JbW11dGFi

This has worked fine up until the day of the server crash. When the server crashed many players lost access to their infinite storages (both their stored locations and actual items stored in it). When I checked the server files I noticed 3 files had normal data stored and 7 player files had their data wiped. The only thing that remained was:
Code: 
name: DOMGODFTW
sort: NORMAL

I'm hiring someone to fix this issue so player's data isn't lost when the server crashes (or only the last few min is at worst) without impacting server performance. I'm guessing this would be done by saving data to file on a schedule(?). However, I am not a plugin developer so I'm not sure if that's correct. Also, any bugs my friend and I find with the plugin after you work on it should also be fixed.

Further discussion of the issue can be found here:
https://www.spigotmc.org/threads/saving-data-of-plugin-into-flatfile-storage.388000/

If you are interested in this job, DM me with the following info:
1. Your portfolio
2. A link to this page
3. How you think the issue could be solved / prevented. Please note that the server was changed from MySQL to Flatfile because of the recurring issues with MySQL.

Payment: $15
Payment Terms: Payment will be sent after I have fully bug tested it on my own server and all bugs have been fixed. As some assurance to you, after I have sent the full payment, please send me the source code (including maven file).
Time Frame: 3 Days

Final Notes:
- Plugin should work for Paper / Spigot 1.14.x+

Not sure if you can update the code? Want to see what the code looks like? Here's an example:
Click to expand...

This may be a result to data corruption. Which method does your plugin use to save player data? Is it locally saved and then transferred to the files on "onDisable" or dynamically updated?
 
InMud7

InMud7

Right where I'm supposed to be ;)
Supreme
Feedback score
1
Posts
154
Reactions
130
Resources
3
Ghast said:
This may be a result to data corruption. Which method does your plugin use to save player data? Is it locally saved and then transferred to the files on "onDisable" or dynamically updated?
Click to expand...
In the file InfiniteStorage.java it says:
Code: 
    /**
     * Called on the plugin disable.
     */

    @Override
    public void onDisable() {
        LocationManager.save();
        UserManager.save();
        this.destroyableHandler.onReload();
        INSTANCE = null;
    }

    @EventHandler
    public void onJoin(PlayerJoinEvent e) {
        UUIDFetcher.cachePlayer(e.getPlayer());
    }
}
 
901cosmic

901cosmic

Have faith in god, work hard, and succeed
Premium
Feedback score
31
Posts
615
Reactions
269
Resources
0
PiggiesGoSqueal said:
In the file InfiniteStorage.java it says:
Code: 
    /**
     * Called on the plugin disable.
     */

    @Override
    public void onDisable() {
        LocationManager.save();
        UserManager.save();
        this.destroyableHandler.onReload();
        INSTANCE = null;
    }

    @EventHandler
    public void onJoin(PlayerJoinEvent e) {
        UUIDFetcher.cachePlayer(e.getPlayer());
    }
}
Click to expand...
That's the main class, try searching for a different class with the name "storage"
 
Ghast

Ghast

Founding Father of Hypocrisy - https://artemis.ac
Supreme
Feedback score
54
Posts
2,096
Reactions
3,285
Resources
79
PiggiesGoSqueal said:
In the file InfiniteStorage.java it says:
Code: 
    /**
     * Called on the plugin disable.
     */

    @Override
    public void onDisable() {
        LocationManager.save();
        UserManager.save();
        this.destroyableHandler.onReload();
        INSTANCE = null;
    }

    @EventHandler
    public void onJoin(PlayerJoinEvent e) {
        UUIDFetcher.cachePlayer(e.getPlayer());
    }
}
Click to expand...
This may be the cause of it. Some sudden crashes do not always take the time to save the data. What I'd suggest is to save the data on every enderchest close event. Not only is this the only most efficient one, but if you put it into parallel streams it should be even better.
Hope I could help,
Cheers,
Ghast
 
YoloSanta

YoloSanta

Bot & Plugin Developer
Supreme
Feedback score
18
Posts
884
Reactions
374
Resources
0
Piggybacking off what ghast said adding something like this should work
Code: 
@EventHandler
    public void event(InventoryCloseEvent event)
    {
       InventoryType type= event.getView().getTopInventory().getType();
       if (type == InventoryType.ENDER_CHEST){
           //save here
       }
    }
 
InMud7

InMud7

Right where I'm supposed to be ;)
Supreme
Feedback score
1
Posts
154
Reactions
130
Resources
3
YoloSanta said:
Piggybacking off what ghast said adding something like this should work
Code: 
@EventHandler
    public void event(InventoryCloseEvent event)
    {
       InventoryType type= event.getView().getTopInventory().getType();
       if (type == InventoryType.ENDER_CHEST){
           //save here
       }
    }
Click to expand...
Lol, I see what you did there. Assuming it was intentional. :p (Piggybacking).

And thanks for your input. :)
 
InMud7

InMud7

Right where I'm supposed to be ;)
Supreme
Feedback score
1
Posts
154
Reactions
130
Resources
3
Still looking for someone. I've had people contact me for my previous post regarding a different plugin but not this one.
 
ZP4RKER

ZP4RKER

Feedback score
17
Posts
227
Reactions
79
Resources
0
If you pass on the advice from Ghast to your original developer, you won't have any need to hire another developer to do it for you. It would be very few lines added to implement that solution.

The onDisable method cannot be guaranteed execution in crashes, hence having the data being saved to file more regularly (like Ghast's suggestion) should eliminate the issue.
 
InMud7

InMud7

Right where I'm supposed to be ;)
Supreme
Feedback score
1
Posts
154
Reactions
130
Resources
3
ZP4RKER said:
If you pass on the advice from Ghast to your original developer, you won't have any need to hire another developer to do it for you. It would be very few lines added to implement that solution.

The onDisable method cannot be guaranteed execution in crashes, hence having the data being saved to file more regularly (like Ghast's suggestion) should eliminate the issue.
Click to expand...
Yeah I just woke up and realized that haha. I'm a bit slow. Thank you for informing me as well.
 
Last edited:
ZP4RKER

ZP4RKER

Feedback score
17
Posts
227
Reactions
79
Resources
0
Status
This thread has been locked.
Top