Thread: Player update protocol - Could use a little guidance :P

Page 1 of 2 12 LastLast
Results 1 to 10 of 17
  1. #1 Player update protocol - Could use a little guidance :P 
    Registered Member
    Join Date
    Feb 2020
    Posts
    5
    Thanks given
    0
    Thanks received
    0
    Rep Power
    0
    Hi all,

    I've been trying to set a default player update state, and I'm pretty sure my frame structure is almost right but it's off sadly. Please excuse the repeated "setBit" methods, I'm just testing an initial login and render with this

    The initial frame 73 works fine, its attempting to load the region. The client isn't picking up the region without the player co-ords though... And I can't give it the co-ords without a player instance lol. So I followed https://rsps.fandom.com/wiki/317_Protocol pretty strictly but no luck . Can anyone help me with the player initialisation procedure? I've put my current bit structure down below! setBit goes by buffer, buffer index, bit index, bit value btw. The total buffer size is byte[13]. And the frame size for frame 81 is 6. So this should work...?

    Just to suffix this, I've never written an RSPS server before, sorry if this is slightly nooby lol.

    Thanks in advance!

    Code:
                            /**
                             * 73: Loads map region for Lumbridge
                             */
                            b[0] = 73 + this.outStreamCryption.getNextKey();
                            b.writeInt16BE(3200, 1);
                            b.writeInt16BE(3200, 3);
    
                            /**
                             * 81: Initialise player
                             */
                            b[5] = 81 + this.outStreamCrypt.nextKey();
                            // update our player or not
                            this.setBit(b, 6, 7, 1);
                            // VALUE: 3 - Type 3, update our players plane level
                            this.setBit(b, 6, 6, 1);
                            this.setBit(b, 6, 5, 1);
                            // setting plane level here, 0-3 (we use 0 for now)
                            this.setBit(b, 6, 4, 0);
                            this.setBit(b, 6, 3, 0);
                            // clear awaiting-point queue, i.e., remove our further steps left to do by client. Like when teleing
                            this.setBit(b, 6, 2, 1);
                            // is there an update required? (i.e., logged in, update our player)
                            this.setBit(b, 6, 1, 1);
                            // x, y co-ords (20x20)
                            this.setBit(b, 6, 0, 0);
                            this.setBit(b, 7, 7, 1);
                            this.setBit(b, 7, 6, 0);
                            this.setBit(b, 7, 5, 0);
                            this.setBit(b, 7, 4, 0);
                            this.setBit(b, 7, 3, 0);
                            this.setBit(b, 7, 2, 0);
    
                            this.setBit(b, 7, 1, 0);
                            this.setBit(b, 7, 0, 1);
                            this.setBit(b, 8, 7, 0);
                            this.setBit(b, 8, 6, 0);
                            this.setBit(b, 8, 5, 0);
                            this.setBit(b, 8, 4, 0);
                            this.setBit(b, 8, 3, 0);
                            // How many other players the client needs to update,
                            // currently no multiplayer so none,
                            // this will need some considerable thought lol
                            this.setBit(b, 8, 2, 0);
                            this.setBit(b, 8, 1, 0);
                            this.setBit(b, 8, 0, 0);
                            this.setBit(b, 9, 7, 0);
                            this.setBit(b, 9, 6, 0);
                            this.setBit(b, 9, 5, 0);
                            this.setBit(b, 9, 4, 0);
                            this.setBit(b, 9, 3, 0);
                            // player list updating, not really sure here, used a recommended 2047 value
                            this.setBit(b, 9, 2, 1);
                            this.setBit(b, 9, 1, 1);
                            this.setBit(b, 9, 0, 1);
                            this.setBit(b, 10, 7, 1);
                            this.setBit(b, 10, 6, 1);
                            this.setBit(b, 10, 5, 1);
                            this.setBit(b, 10, 4, 1);
                            this.setBit(b, 10, 3, 1);
                            this.setBit(b, 10, 2, 1);
                            this.setBit(b, 10, 1, 1);
                            this.setBit(b, 10, 0, 1);
                            // write frame size in short format (6 byte)
                            this.setBit(b, 12, 2, 1);
                            this.setBit(b, 12, 1, 1);
                            this.setBit(b, 12, 0, 0);
                            // initial player update done
                            socket.write(b);
    Reply With Quote  
     

  2. #2  
    Programmer, Contributor, RM and Veteran




    Join Date
    Mar 2007
    Posts
    5,147
    Thanks given
    2,656
    Thanks received
    3,731
    Rep Power
    5000
    Code:
                            b[0] = 73 + this.outStreamCryption.getNextKey();
                            b.writeInt16BE(3200, 1);
                            b.writeInt16BE(3200, 3);
    This packet contains the coordinates of the central zone (an 8x8 group of tiles), not the central tile. Try dividing these numbers by 8.

    Another problem is that the first short in the packet has the "A" modifier (idk if we have a good name for these), so it's encoded like so (using the code from wL):

    Code:
        public void writeWordA(int i)
        {
            buffer[currentOffset++] = (byte)(i >> 8);
            buffer[currentOffset++] = (byte)(i + 128);
        }
    note the extra "+ 128". The client undoes this on the other end.

    So you'll probably want to make a new writeInt16BEA method for this. (Other similar obfuscation used by Jagex includes the "C" and "S" modifiers and middle-endian integers.)

    I'd strongly suggest writing some sort of buffer type (or using an existing one) that keeps track of the index you're currently reading or writing, so you don't need to track this yourself. Your bit packing code is very hard to read in particular, and calculating all those indexes manually is bound to be error-prone.

    What does the client do when you send that packet? Does it crash? If so, the client-side error is useful. If it doesn't, describing what it does do (e.g. do you just see a black screen?) will also be useful.

    Code:
                            b[5] = 81 + this.outStreamCrypt.nextKey();
                            // update our player or not
                            this.setBit(b, 6, 7, 1);
    you need to leave two bytes after the opcode for the packet length, as the player update packet has a variable short length. So you should be starting to set bits at index 8, not 6.

    You'll also need to fill in this length with the number of bytes in the update packet.
    .
    Reply With Quote  
     

  3. Thankful users:


  4. #3  
    Registered Member
    Join Date
    Feb 2020
    Posts
    5
    Thanks given
    0
    Thanks received
    0
    Rep Power
    0
    Quote Originally Posted by Graham View Post
    Code:
                            b[0] = 73 + this.outStreamCryption.getNextKey();
                            b.writeInt16BE(3200, 1);
                            b.writeInt16BE(3200, 3);
    This packet contains the coordinates of the central zone (an 8x8 group of tiles), not the central tile. Try dividing these numbers by 8.

    I'd strongly suggest writing some sort of buffer type (or using an existing one) that keeps track of the index you're currently reading or writing, so you don't need to track this yourself. Your bit packing code is very hard to read in particular, and calculating all those indexes manually is bound to be error-prone.

    What does the client do when you send that packet? Does it crash? If so, the client-side error is useful. If it doesn't, describing what it does do (e.g. do you just see a black screen?) will also be useful.
    Hey Graham,

    I thought this was the region? I've using this thing: https://explv.github.io/ lol. So it's the 8x8 group?

    And I thought the central tile co-ordinate was in P81? The 317 protocol doc I'm using doesn't explain too much. Do you know of any better resources out there?

    And I am gonna write two buffer readers.

    No client errors sadly, just sits on "Loading - please wait.".

    EDIT:

    Quote Originally Posted by Graham View Post

    Code:
                            b[5] = 81 + this.outStreamCrypt.nextKey();
                            // update our player or not
                            this.setBit(b, 6, 7, 1);
    you need to leave two bytes after the opcode for the packet length, as the player update packet has a variable short length. So you should be starting to set bits at index 8, not 6.

    You'll also need to fill in this length with the number of bytes in the update packet.
    Ohhh? I've put the packet size at the end lol. Lke a frame header/footer setup?
    EDIT:
    Like this:
    Code:
                            this.setBit(b, 10, 0, 1);
                            // write frame size in short format
                            // size is 40 bits
                            // we upto byte10 used, we on [11] now
                            this.setBit(b, 12, 2, 1);
                            this.setBit(b, 12, 1, 1);
                            this.setBit(b, 12, 0, 0);
    buf[11] byte is 0, buf [12] only write these 3, = 6

    Quote Originally Posted by Graham View Post
    Another problem is that the first short in the packet has the "A" modifier (idk if we have a good name for these), so it's encoded like so (using the code from wL):
    Oooh, thanks. I'll give this a go later! I bet that's the issue, thank youuuu
    Reply With Quote  
     

  5. #4  
    Programmer, Contributor, RM and Veteran




    Join Date
    Mar 2007
    Posts
    5,147
    Thanks given
    2,656
    Thanks received
    3,731
    Rep Power
    5000
    I thought this was the region? I've using this thing: https://explv.github.io/ lol. So it's the 8x8 group?
    Yeah. (3200, 3200) is the position of the tile, and using that map viewer to find those is fine. But that packet takes the position of the zone, not tile, so you need to divide everything by 8.

    A quick coordinate system reference:

    8x8 tiles = a zone. This is the granularity used in a bunch of packets (e.g. for sending ground items, projectiles, constructing dynamic/instanced maps, etc.)
    64x64 tiles (or 8x8 zones) = a single map. This is the granularity used on disk in the cache.
    104x104 tiles (or 13x13 zones) = the area that the client keeps in RAM at any one time. You'll also find +/- 6 calculations in some servers (and perhaps the client too) - this is because some code uses zone coordinates relative to the top left of the area, some relative to the centre. When the player is within 16 tiles of the edge the server sends the packet to reload the area around the player's current zone.

    The entire world is 16384x16384 tiles (so 256x256 maps or 2048x2048 zones). The left half (8192x16384 tiles) is for static maps (the ones used the majority of the time). The right half is used for maps created dynamically at runtime, like player owned houses, instanced areas, etc.

    Ohhh? I've put the packet size at the end lol. Lke a frame header/footer setup?
    Yes, the packet length is in the header.

    Fixed length packets just have a single byte header - the opcode.
    Variable length packets have the opcode followed by either a byte or short for the length.

    In all cases this is then followed by the payload. There is no trailer.
    .
    Reply With Quote  
     

  6. Thankful user:


  7. #5  
    Registered Member
    Join Date
    Feb 2020
    Posts
    5
    Thanks given
    0
    Thanks received
    0
    Rep Power
    0
    Quote Originally Posted by Graham View Post
    Yeah. (3200, 3200) is the position of the tile, and using that map viewer to find those is fine. But that packet takes the position of the zone, not tile, so you need to divide everything by 8.

    A quick coordinate system reference:

    8x8 tiles = a zone. This is the granularity used in a bunch of packets (e.g. for sending ground items, projectiles, constructing dynamic/instanced maps, etc.)
    64x64 tiles (or 8x8 zones) = a single map. This is the granularity used on disk in the cache.
    104x104 tiles (or 13x13 zones) = the area that the client keeps in RAM at any one time. You'll also find +/- 6 calculations in some servers (and perhaps the client too) - this is because some code uses zone coordinates relative to the top left of the area, some relative to the centre. When the player is within 16 tiles of the edge the server sends the packet to reload the area around the player's current zone.

    The entire world is 16384x16384 tiles. The left half (so 8192x16384 tiles) is for static maps (the ones used the majority of the time). The right half is used for maps created dynamically at runtime, like player owned houses, instanced areas, etc.



    Yes, the packet length is in the header.

    Fixed length packets just have a single byte header - the opcode.
    Variable length packets have the opcode followed by either a byte or short for the length.

    In all cases this is then followed by the payload. There is no trailer.
    Ahh, my mistake. I'm a web dev, not game dev and usually when I've worked with TCP binary, we put sizes in footer.

    Oh wow, ok ok, gotcha! Thank you loads man. Appreciate it, how did you ever find this small details? In all clients I downloaded that are deob'd, everything is still unnamed and impossible to read lol and when I've tried to search for resources, there's not much .

    EDIT: So the 8x8 short is read and written in little endian then? Else how we + 128?

    EDIT: Thanks Graham on the player updating, all good now. As for the map region though, no luck. Little bit confused on the endian format to write in: https://rsps.fandom.com/wiki/317_Protocol gives no identifying info. I'll continue to mess with BE'A' and LE'A' and manually write one soon to see if it works.

    Can fully confirm all update types are read as BE. Thanks Graham for the help! Really appreciate it!

    Attached image


    Think I need more help Graham .

    Is the appearance updating apart of packet 81?
    https://rsps.fandom.com/wiki/317_Protocol#Stage_1:_Client_-.3E_Server
    This really isn't telling me much. Also, in order to get the initial part of packet 81 to work, I had to suffix it with an empty byte right after player list updating, any idea why?
    Reply With Quote  
     

  8. #6  
    Programmer, Contributor, RM and Veteran




    Join Date
    Mar 2007
    Posts
    5,147
    Thanks given
    2,656
    Thanks received
    3,731
    Rep Power
    5000
    Is the appearance updating apart of packet 81?
    Yep

    Also, in order to get the initial part of packet 81 to work, I had to suffix it with an empty byte right after player list updating, any idea why?
    Probably the field containing the number of players currently visible.

    The rough overall structure of the packet is:

    Code:
    * bit-packed section
      * movement for current player (idle, walk, run or teleport)
      * number of existing local players (other than the current player)
      * for each existing local player: movement (idle, walk, run) or removal
      * addition of new local players - note that teleporting (other than the current player) is implemented by removing and then re-adding a player
    * byte-packed section (optional, the new player list is terminated with an id of 2047 if present)
      * update blocks for the current player
      * update blocks for existing local players that were not removed
      * update blocks for new local players
    Each update block starts with a flags indicating the type of block that follow (e.g. appearance, chat message, animation, etc.) and then the actual data for each block.

    The update blocks are only present if an earlier bit is set within the bit-packed section.

    The easiest way to implement this is to encode the bit-packed buffer at the same time as the block updates and then join the two buffers together at the end.
    .
    Reply With Quote  
     

  9. Thankful users:


  10. #7  
    Registered Member
    Join Date
    Sep 2019
    Posts
    29
    Thanks given
    2
    Thanks received
    8
    Rep Power
    65
    Quote Originally Posted by Graham View Post
    Yep



    Probably the field containing the number of players currently visible.

    The rough overall structure of the packet is:

    Code:
    * bit-packed section
      * movement for current player (idle, walk, run or teleport)
      * number of existing local players (other than the current player)
      * for each existing local player: movement (idle, walk, run) or removal
      * addition of new local players - note that teleporting (other than the current player) is implemented by removing and then re-adding a player
    * byte-packed section (optional, the new player list is terminated with an id of 2047 if present)
      * update blocks for the current player
      * update blocks for existing local players that were not removed
      * update blocks for new local players
    Each update block starts with a flags indicating the type of block that follow (e.g. appearance, chat message, animation, etc.) and then the actual data for each block.

    The update blocks are only present if an earlier bit is set within the bit-packed section.

    The easiest way to implement this is to encode the bit-packed buffer at the same time as the block updates and then join the two buffers together at the end.
    Brill, thanks man! (I'm on my account now was using my gf's as I forgot my pwd lol). I've dove into the client and found the method143 call, with it all in. Just figured the rest of the packet out right before I read this but thank you anyways hehe. As for update block flags, super insightful. Tried reading method49 in the client and it honestly blew my head off haha. Really appreciate this, thank you!
    Reply With Quote  
     

  11. #8  
    Registered Member
    Join Date
    Sep 2019
    Posts
    29
    Thanks given
    2
    Thanks received
    8
    Rep Power
    65
    Quote Originally Posted by Graham View Post
    Yep



    Probably the field containing the number of players currently visible.

    The rough overall structure of the packet is:

    Code:
    * bit-packed section
      * movement for current player (idle, walk, run or teleport)
      * number of existing local players (other than the current player)
      * for each existing local player: movement (idle, walk, run) or removal
      * addition of new local players - note that teleporting (other than the current player) is implemented by removing and then re-adding a player
    * byte-packed section (optional, the new player list is terminated with an id of 2047 if present)
      * update blocks for the current player
      * update blocks for existing local players that were not removed
      * update blocks for new local players
    Each update block starts with a flags indicating the type of block that follow (e.g. appearance, chat message, animation, etc.) and then the actual data for each block.

    The update blocks are only present if an earlier bit is set within the bit-packed section.

    The easiest way to implement this is to encode the bit-packed buffer at the same time as the block updates and then join the two buffers together at the end.

    Quick question Graham, when updating for new players and our local player has a bitmask appended, how can I avoid the loop (for the player list updating) including the masks? It's running ovber the entire packet size, i.e., reading masks for our/other player updates.

    See, for a single player the 2047 clause will work as we know the read synchronously across the board afterwards, but if this wasn't the case, this loop is going to run for this length:

    Code:
    Movement type set to: 3
    Amount of other player movements to sync: 0
    Buffer bit position check (+10): 39 and packet size (bits) is: 504
    Player list loop ran 1 times and bit position is: 40
    Mask id is: 16
    39 -> 504, and each loop it only runs a total of 23 bits per player. My mask is included in the above... So if I sent another other players index and let's say they've got an updatemask too, that'll be well over 1000 bits for 1 (2 really cause of us but we don't handle us here I know, our update index added in the local movement) player(s) which should only be 70 or so bits (1 loop for 1 other player).
    Code:
    while (buffer.getBitPosition() + 10 < packetSize * 8) {
    I'm honestly stumped here lol. What is the process for introducing new players? I can't just simply send their index sadly lol.
    Reply With Quote  
     

  12. #9  
    Registered Member

    Join Date
    Feb 2010
    Posts
    3,253
    Thanks given
    1,145
    Thanks received
    909
    Rep Power
    2081
    Quote Originally Posted by Graham View Post
    Code:
                            b[0] = 73 + this.outStreamCryption.getNextKey();
                            b.writeInt16BE(3200, 1);
                            b.writeInt16BE(3200, 3);
    This packet contains the coordinates of the central zone (an 8x8 group of tiles), not the central tile. Try dividing these numbers by 8.

    Another problem is that the first short in the packet has the "A" modifier (idk if we have a good name for these), so it's encoded like so (using the code from wL):

    Code:
        public void writeWordA(int i)
        {
            buffer[currentOffset++] = (byte)(i >> 8);
            buffer[currentOffset++] = (byte)(i + 128);
        }
    note the extra "+ 128". The client undoes this on the other end.

    So you'll probably want to make a new writeInt16BEA method for this. (Other similar obfuscation used by Jagex includes the "C" and "S" modifiers and middle-endian integers.)

    I'd strongly suggest writing some sort of buffer type (or using an existing one) that keeps track of the index you're currently reading or writing, so you don't need to track this yourself. Your bit packing code is very hard to read in particular, and calculating all those indexes manually is bound to be error-prone.

    What does the client do when you send that packet? Does it crash? If so, the client-side error is useful. If it doesn't, describing what it does do (e.g. do you just see a black screen?) will also be useful.

    Code:
                            b[5] = 81 + this.outStreamCrypt.nextKey();
                            // update our player or not
                            this.setBit(b, 6, 7, 1);
    you need to leave two bytes after the opcode for the packet length, as the player update packet has a variable short length. So you should be starting to set bits at index 8, not 6.

    You'll also need to fill in this length with the number of bytes in the update packet.
    Doesn't the a just stand for Add because you are adding 128 and s is subtracting 128?
    Reply With Quote  
     

  13. #10  
    Programmer, Contributor, RM and Veteran




    Join Date
    Mar 2007
    Posts
    5,147
    Thanks given
    2,656
    Thanks received
    3,731
    Rep Power
    5000
    Quote Originally Posted by wolfajk View Post
    Quick question Graham, when updating for new players and our local player has a bitmask appended, how can I avoid the loop (for the player list updating) including the masks? It's running ovber the entire packet size, i.e., reading masks for our/other player updates.

    See, for a single player the 2047 clause will work as we know the read synchronously across the board afterwards, but if this wasn't the case, this loop is going to run for this length:

    Code:
    Movement type set to: 3
    Amount of other player movements to sync: 0
    Buffer bit position check (+10): 39 and packet size (bits) is: 504
    Player list loop ran 1 times and bit position is: 40
    Mask id is: 16
    39 -> 504, and each loop it only runs a total of 23 bits per player. My mask is included in the above... So if I sent another other players index and let's say they've got an updatemask too, that'll be well over 1000 bits for 1 (2 really cause of us but we don't handle us here I know, our update index added in the local movement) player(s) which should only be 70 or so bits (1 loop for 1 other player).
    Code:
    while (buffer.getBitPosition() + 10 < packetSize * 8) {
    I'm honestly stumped here lol. What is the process for introducing new players? I can't just simply send their index sadly lol.
    I don't quite understand what you're trying to get at. The magic 2047 clause works for any number of players - it makes the client break out of the loop early and switch to the block reading mode.

    Remember that the bit-based updates for all players are always prior to the (byte-based) blocks for all players. They aren't interleaved - i.e: the order is like this:

    - existing player 1 bits
    - existing player 2 bits
    - new player 3 bits
    - magic 2047 marker
    - existing player 1 blocks
    - existing player 2 blocks
    - new player 3 blocks

    If no players have any blocks, then you can omit the magic 2047 marker, because there is no need to distinguish between bits/blocks in that case. The client detects it is at the end of the packet and skips reading the blocks.

    Quote Originally Posted by Fire Cape View Post
    Doesn't the a just stand for Add because you are adding 128 and s is subtracting 128?
    The A/C/S suffixes aren't Jagex's naming scheme (I don't think we know what Jagex call them).

    But yes, I've long suspected that whoever used those suffixes originally used them to stand for add, complement and subtract respectively.
    .
    Reply With Quote  
     

Page 1 of 2 12 LastLast

Thread Information
Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)


User Tag List

Similar Threads

  1. [731] Player updating protocol
    By clem585 in forum Help
    Replies: 5
    Last Post: 06-22-2018, 11:47 AM
  2. Could Use a Little Help; 718
    By W R X in forum Help
    Replies: 0
    Last Post: 03-22-2015, 07:44 PM
  3. Replies: 6
    Last Post: 02-22-2013, 11:09 PM
  4. Replies: 36
    Last Post: 02-28-2012, 12:59 AM
  5. Replies: 8
    Last Post: 03-11-2008, 01:19 PM
Posting Permissions
  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •