Thread: Universal Base Client Idea - It's probably been suggested before though

Results 1 to 8 of 8
  1. #1 Universal Base Client Idea - It's probably been suggested before though 
    Registered Member ColdBlooded's Avatar
    Join Date
    Sep 2017
    Posts
    41
    Thanks given
    3
    Thanks received
    1
    Rep Power
    0
    Why not make a client that packs the cache in a 7zip, and/or zip (perhaps .cab) file, and no map files. The zip file would require way less code to unpack, and maps can already be made using map-editors. Just the game engine itself. It could save a lot of servers from getting complaints from jagex if obbed properly. Would also be easier to update, add, and remove things without the need of those cache tools. Almost certain it's been suggested before, but why not?
    Attached image
    Reply With Quote  
     

  2. #2  
    ???

    funkE's Avatar
    Join Date
    Feb 2008
    Posts
    2,612
    Thanks given
    255
    Thanks received
    989
    Rep Power
    1366
    If your problem is finding tools to work in the filesystem, and tools to edit files in it, the solution is better tools that understand more formats.

    If you're so concerned about complaints, go to a place that doesn't listen. You want to engineer some way out of a problem that will still exist regardless of what you do: it's Jagex's game, their content.

    Choosing a host in another country: easy
    Rewriting the entire game and probably not accomplishing any goal: hard

    Idk man, sounds like a good idea when you're sitting on the toilet but it doesn't really make sense.
    .
    Reply With Quote  
     

  3. Thankful user:


  4. #3  
    Respected Member


    Join Date
    Jan 2009
    Posts
    5,743
    Thanks given
    1,162
    Thanks received
    3,603
    Rep Power
    5000
    with zip you'd typically be loading the entire thing into memory
    Reply With Quote  
     

  5. #4  
    SERGEANT OF THE MASTER SERGEANTS MOST IMPORTANT PERSON OF EXTREME SERGEANTS TO THE MAX!

    cube's Avatar
    Join Date
    Jun 2007
    Posts
    8,871
    Thanks given
    1,854
    Thanks received
    4,745
    Rep Power
    5000
    No one is stopping you from using an extracted cache for the server, the packed client version shouldn't be modified by anything else than the client.
    If this thread is about the file formats then it's pointless as almost all formats are best edited with editors and converting rs-formats to text based would make the cache many times bigger

    Attached image

    Reply With Quote  
     

  6. #5  
    Registered Member ColdBlooded's Avatar
    Join Date
    Sep 2017
    Posts
    41
    Thanks given
    3
    Thanks received
    1
    Rep Power
    0
    Quote Originally Posted by S Quare Quxx View Post
    No one is stopping you from using an extracted cache for the server, the packed client version shouldn't be modified by anything else than the client.
    If this thread is about the file formats then it's pointless as almost all formats are best edited with editors and converting rs-formats to text based would make the cache many times bigger
    its partially about file formats. i understand that having thousands of png or .dat files makes the cache large, though, there's probably better compressors out now that work with the cache a bit better than the jagex format. it's mostly about a custom way of storing files so that it's not identical to the way jagex does it, so it doesn't infringe copyright

    Quote Originally Posted by Stuart View Post
    with zip you'd typically be loading the entire thing into memory
    i appreciate the bluntness. couldn't it load it by chunks or several zip files? perhaps 7zip, although the decompression would prob take a while
    Attached image
    Reply With Quote  
     

  7. #6  
    ???

    funkE's Avatar
    Join Date
    Feb 2008
    Posts
    2,612
    Thanks given
    255
    Thanks received
    989
    Rep Power
    1366
    Quote Originally Posted by ColdBlooded View Post
    its partially about file formats. i understand that having thousands of png or .dat files makes the cache large, though, there's probably better compressors out now that work with the cache a bit better than the jagex format. it's mostly about a custom way of storing files so that it's not identical to the way jagex does it, so it doesn't infringe copyright
    It's still Jagex assets, it will always infringe copyright. There are better ways to solve the problem than trying to engineer a new file format to beat the system. RuneScape appends to the filestore as it downloads data, so everyone's cache is different once they start downloading and going to different areas and seeing new models. They aren't necessarily busting people by doing checksum matches against files (unless you host a zip, lol)

    Jagex uses gzip and bzip which have super great compression rates, all their formats are already as small and optimized as can be. You won't get any smaller. RuneScape, as a piece of technology, is very well engineered to deal with the dialup systems of way-back-when. It's wonderful. You will not gain much, and you'll spend so much time on something that does not achieve your goals, so why even bother?

    i appreciate the bluntness. couldn't it load it by chunks or several zip files? perhaps 7zip, although the decompression would prob take a while
    Just use the cache, my man, it does this already. You will not defeat anything by changing the storage format.

    If you're worried about distribution and take down notices you have a few things you can do:
    1. Let the server send the data to the client the way it was intended (Done)
    2. Modify the JAGGRAB and OnDemand protocols to use encryption (Easy)
    3. Utilize proxies sitting in front of your servers so your actual host is never known (Easy)
    4. Don't distribute the cache as a single file, like a zip (Done)
    5. Host everything in a country that doesn't give a shit (Easy)


    There are the better ideas! Don't get attached to your idea, it doesn't work and won't ever work the way you're envisioning!!!
    .
    Reply With Quote  
     

  8. Thankful user:


  9. #7  
    Registered Member ColdBlooded's Avatar
    Join Date
    Sep 2017
    Posts
    41
    Thanks given
    3
    Thanks received
    1
    Rep Power
    0
    Quote Originally Posted by funkE View Post
    It's still Jagex assets, it will always infringe copyright. There are better ways to solve the problem than trying to engineer a new file format to beat the system. RuneScape appends to the filestore as it downloads data, so everyone's cache is different once they start downloading and going to different areas and seeing new models. They aren't necessarily busting people by doing checksum matches against files (unless you host a zip, lol)

    Jagex uses gzip and bzip which have super great compression rates, all their formats are already as small and optimized as can be. You won't get any smaller. RuneScape, as a piece of technology, is very well engineered to deal with the dialup systems of way-back-when. It's wonderful. You will not gain much, and you'll spend so much time on something that does not achieve your goals, so why even bother?



    Just use the cache, my man, it does this already. You will not defeat anything by changing the storage format.

    If you're worried about distribution and take down notices you have a few things you can do:
    1. Let the server send the data to the client the way it was intended (Done)
    2. Modify the JAGGRAB and OnDemand protocols to use encryption (Easy)
    3. Utilize proxies sitting in front of your servers so your actual host is never known (Easy)
    4. Don't distribute the cache as a single file, like a zip (Done)
    5. Host everything in a country that doesn't give a shit (Easy)


    There are the better ideas! Don't get attached to your idea, it doesn't work and won't ever work the way you're envisioning!!!
    thanks for putting in the time into explaining this well. it makes a lot more sense now. it would still be nice to have a dev client and a release client, where the dev client has everything in raw format for easy editing, including the landscape and object maps, as well as, sprites and other compressed data, then the release client would contain the compressed form. but i'm aware that's the reason why cache editors exist lol.
    Attached image
    Reply With Quote  
     

  10. #8  
    ???

    funkE's Avatar
    Join Date
    Feb 2008
    Posts
    2,612
    Thanks given
    255
    Thanks received
    989
    Rep Power
    1366
    Quote Originally Posted by ColdBlooded View Post
    thanks for putting in the time into explaining this well. it makes a lot more sense now. it would still be nice to have a dev client and a release client, where the dev client has everything in raw format for easy editing, including the landscape and object maps, as well as, sprites and other compressed data, then the release client would contain the compressed form. but i'm aware that's the reason why cache editors exist lol.
    You're welcome, I GUESS the intention of R-S was originally supposed to be a place where people can come to share ideas, collaborate, and learn. Someone like you, who is making an honest effort to find+address+innovate+discuss problems, deserves to be treated well.


    I don't think their tools use different formats. I'd say that good developers focus on reducing complexity, only introducing it for justifiable reasons. Sure, the content of the game is questionable, but the game code is very well crafted.

    I just can't help but question: why make an intermediate format? Nobody is going to be editing the files, the editor will be doing that saving. The file formats are VERY good. Afaik the client they use for editing uses the same core as the game so it could go a few ways, but I don't think any of them would involve an intermediate format.

    Speculation:
    Maybe they work on the data as actual files and only use the cache system for the client. That would mean the ONDEMAND/JAGGRAB servers dish out real files and the client receives them, then stores them into the archive format. There's no way to really know, but it's plausible.

    The cache is like a stripped down, proprietary archive format. It can be thought of like a zip, tar, etc. except it has been purpose-built for RS. Decompressing the files every time you need to serve them seems inefficient for Jagex to do... I admit I don't know how big the cache is when it has been expanded into real directories and files.

    A sort of relevant idea:
    I've always wanted to implement was a file protocol implementation for the game files to make server configuration and client editing easier.
    If you decouple the editor from the storage medium then you have the freedom to use real files or files within the archive (cache).
    So something like a handler for cache://cache_dir/index/file so you can specify exactly what directory to use and the protocol handler can automatically say:
    "Does main_file_cache.dat exist?" No -> expanded files, Yes -> archive files
    See Michael Remijan: Create Your Own Java URL Handlers and https://stackoverflow.com/questions/...t-url-protocol for more info on this idea

    A solution like this also means that your protocol does not change, so you can swap out for support of higher or lower revision caches without the protocol changing, and without requiring the editing code to care about what version format the archive is. All your tools can use this format and you can add support for more archive format versions.
    If you want to detect the version of the cache you would be able to do something like cache://cache_dir/info to receive some structured data (yaml, json, xml, whatever) of all the information you could possibly want to know about it
    Indices w/ counts, checksums, format version, etc. so your editor can know what the format of the files you're editing automatically

    I'd really love to innovate in the tools department, if you couldn't tell already
    Oh, and also creating a standalone solution for OnDemand/JAGGRAB and adding support for encryption into the protocols while still being backwards compatible with clients that don't support the encryption. E-Z
    Last edited by funkE; 04-08-2018 at 07:43 PM.
    .
    Reply With Quote  
     

  11. Thankful user:



Thread Information
Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)


User Tag List

Similar Threads

  1. Replies: 4
    Last Post: 06-15-2014, 05:10 AM
  2. New Project Base & Client Ideas
    By Equity in forum Help
    Replies: 7
    Last Post: 09-10-2013, 10:52 PM
  3. Project Different: Base Client
    By Mrthunder23 in forum Downloads
    Replies: 19
    Last Post: 02-10-2009, 11:53 PM
  4. Need client ideas BAD
    By ZachHaley in forum RS2 Client
    Replies: 34
    Last Post: 05-21-2008, 09:50 PM
  5. Dev Kit + a working silab client in it..
    By Ownagepkz in forum Downloads
    Replies: 6
    Last Post: 10-04-2007, 04:16 PM
Posting Permissions
  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •