|
Although I'd recommend to stick to JSON, you could further optimise your binary storage format by only storing non-zero values, whereas currently, if you have the following bonuses:
{ 0, 0, 0, 0, 0, 0, 1, 2 }
you would be storing 8 shorts, which could and should be optimised to something like this (which would only require storing 2 + 1 shorts).
The code below could be ported from Java and arbitrarily relies upon Netty's ByteBuf implementation:
N.B. if the size of the bonuses array is a fixed length, you could remove a few additional bytes from each definition!Code:private static class ItemDefinition { private final int id; private final String name; private final String description; private final short[] bonuses; public void encode(ByteBuf buffer) { buffer.writeShort(id); buffer.writeShort(name.length()); buffer.writeCharSequence(name, StandardCharsets.UTF_8); buffer.writeShort(description.length()); buffer.writeCharSequence(description, StandardCharsets.UTF_8); if (bonuses == null) buffer.writeShort(0); else { short mask = 0; for (short index = 0; index < bonuses.length; index++) { if (bonuses[index] != 0) mask |= (short) 1 << index; } buffer.writeShort(mask); if (mask != 0) { buffer.writeByte(bonuses.length); for (short index = 0; index < bonuses.length; index++) { if (bonuses[index] != 0) buffer.writeShort(bonuses[index]); } } } } ItemDefinition(int id, String name, String description, short[] bonuses) { this.id = id; this.name = name; this.description = description; this.bonuses = bonuses; } public static ItemDefinitionBuilder builder() { return new ItemDefinitionBuilder(); } public int getId() { return id; } public String getName() { return name; } public String getDescription() { return description; } public short[] getBonuses() { return bonuses; } public static class ItemDefinitionBuilder { private int id; private String name; private String description; private short[] bonuses; ItemDefinitionBuilder() {} public ItemDefinition.ItemDefinitionBuilder id(int id) { this.id = id; return this; } public ItemDefinition.ItemDefinitionBuilder name(String name) { this.name = name; return this; } public ItemDefinition.ItemDefinitionBuilder description(String description) { this.description = description; return this; } public ItemDefinition.ItemDefinitionBuilder bonuses(short[] bonuses) { this.bonuses = bonuses; return this; } public ItemDefinition.ItemDefinitionBuilder decode(ByteBuf buffer) { id(buffer.readShort()); name(buffer.readCharSequence(buffer.readShort(), StandardCharsets.UTF_8).toString()); description(buffer.readCharSequence(buffer.readShort(), StandardCharsets.UTF_8).toString()); short mask = buffer.readShort(); if (mask != 0) { bonuses = new short[buffer.readByte()]; for (short index = 0; index < bonuses.length; index++) { if ((mask & ((short) 1 << index)) != 0) bonuses[index] = buffer.readShort(); } } return this; } public ItemDefinition build() { return new ItemDefinition(id, name, description, bonuses); } } }
your favorite project is back (us too dont forget)
https://gfycat.com/LeafyFemaleDiamondbackrattlesnake
New features:
- Logging in
- Correct name-hashes (names used to be fucked, incorrect name -> long -> name procedure)
- Appearance block writing
- Movement (lol)
Current goals:
- Proper appearance-block writing (writes every tick, cant get it to change genders fucking cisgendered misogynistic intolerant trump supporting client)
- Player updating
- PROPER movement (and running/walking)
and why would I stop! Updates on the thread are a little slow because lots of stuff going on IRL, but if you check the bitbucket updates are much more frequent!
Baby update (compared to what is coming soon )
- Itemdefs moved from 459 to OSRS 145
- Lots of params added and put into json (is_2h, is_noted, etc). This is going to be for ease, rather than handling this stuff in the code, we can just put it in itemdef! Makes python more viable for server hosting
- JSON file size - 1077kb -> 3977kb
- New binary pack format (8.82% size of original, retains all relevant data, with a caveat)
- New binary is 55.4% size of the old binary and loads even faster (thanks, caveat)
- Nulls are not fully packed, their data is all the same.
Info on new binary packing format:
Code:Item binary data length - 1 byte -- Length of the binary data for any given itemdef (will probably be changed to more if I decide to get item descriptions) Item ID - 15 bits - If the item is a null, the ID written will be 32767 and if a 32767 is read, go to the next item. Item name length - 6 bits - This is because the longest item name contains 35 characters Item Name - 7 bits per char - [Lines 88-99, 145-146, 189] CAVEAT - Nothing for item description. Right now all item descriptions are "It's a(n) xxx." so this can be eliminated while packing binary. If I ever bother to scrape the descriptions, they will be handled like item names. Item Price - 31 bits - This is for a max value of ~2147m, you won't be selling items for a negative price or for more than max-stack of an item, so last bit isnt' necessary in either scenario Item is 2h - 1 bit - True/False Item is stackable - 1 bit - True/False Item is tradable - 1 bit - True/False Item is sellable - 1 bit - True/False Item is droppable - 1 bit - True/False Item is noted - 1 bit - True/False Item can note - 1 bit - True/False If item isnt noted and item can note: Item noted id - 15 bits - This is the noted item id Item bonuses - 1-10 bits (first 4 for bonus id 0-13, last 9 for value -255 to 255). If a 1 is read, next 9 bits are read. If a 0 is read, the next bit is read for checking if there is a bonus After the 13th bit check (and possibly bonus grab), go to next item if last bit written is the last bit of the last byte: a 255 byte is added (11111111) if last bit written is not the last bit of the last byte (if last byte was written with 1-7 bits): 1s are written to the rest of the byte and the byte's bits are reversed this is to maintain data integrity
Update - cache loading
It's been slow because school started. (Also switched to OSRS protocol).
Currently loading revision 152 cache (ty openrs).
No CRC calculating yet, but want to finish it ASAP because protocol requires it.
Link to latest Bitbucket commit 15c0671 - Renamed sector/cachepointer properties
Nice progress man.
Looks good man
I know this is a grave dig, bug er. Any chance this framework is still being updated? Python is great
« Previous Thread | Next Thread » |
Thread Information |
Users Browsing this ThreadThere are currently 1 users browsing this thread. (0 members and 1 guests) |
Tags for this Thread |