13 replies to this topic

[Jon Cunningham]

Introduction

On our backburner has been a plan to offer an API that allows third-party devs to make tools and websites about MechWarrior Online. Currently, these tools are made only through the rather laborious process of manually extracting XML files and reading them by hand.

This is not ideal.

Ideally we want to offer an easy to use, always-up-to-date and centralized source of information for people building online tools.

The first step in that process is defining standardized, easy-to-use data structures for third party tools. We've chosen JSON as a standard format. JSON objects are valid javascript "objects" that can be used directly without any special parsing on websites. They are also easy to parse into arrays for other languages, like PHP.

You can see the beginning of our thoughts about how the data should be physically structured inside these files (they're hosted on the CDN, so don't worry about overwhelming the website by opening them).

Items
http://static.mwomer.../list/full.json

Omniparts
http://static.mwomer.../list/full.json

Mechs
http://static.mwomer.../list/full.json
http://static.mwomer.../list/dict.json
http://static.mwomer...view/kfx-s.json
(or any mech code)

What's next?

We plan to offer a less verbose mech file that lists only part IDs rather than the entire part in the loadout section, and also a version that has been minified.

Some of the data and keys will be normalized so that it's more predictable what you need to type in order to read a part.

We'll finalize the location of these files at some point.

Possibly in the future, some short tutorials on how to use these files will be created.


A FAQ to some common concerns:

Will these files give me a "sneak peak" of upcoming changes to the game?

No - the data included here is automatically generated from already-available data in the game client. There's nothing new or special in these files that wasn't already available to you.

However they are a *lot* easier to read. Even if you don't program, you can pop open these files and read about stuff in the game. I'll happily answer any questions you have about how certain properties work.

About the design of the API...

You include redundant data from time to time, like having setting the chassis value set at both the mech level and on individual parts. Why?

To add support for MVVM-style frameworks which make use of object graphs. An individual part may or may not be an omnipart, which means it is removable. As a removable model, it would require that information, so we supply it on all parts.



Some parts have values that are set to null, 0 or negative one. why even include the values, then?

We're aiming for the data to be reasonably complete, and we want to shy away from specialized values on parts and modules when they should be generalized. For instance, internals are set to a value of 0 tons. It is somewhat easier to simply loop "all" parts for tonnage values. A specific benefit is that if we ever add something that uses the internal system in the game but has actual mass, we don't break any code that presumes all internals to be 0 mass.



A lot of stock mechs report total tonnage at just under max. Why?

The armour system in the game actually uses percentages for boosting armour points per ton, rather than flat integers like 16, 18 & 19. The game UI hides the actual value of the tonnage by rounding up at a certain point, even though they are not technically at their maximum tonnage. We thought it would be best to accurately report their real values here.


You deviate from the XML in many places, like for instance you do not specify rears as distinct limbs in the data.

Yes. As much as possible we try to "collapse" XML files into single objects, so that a mech is fully and entirely defined by a central object. Requiring developers to read several different files just to learn whether a a mech may legally place a large laser in the left torso is counterproductive.

In addition we also try to eliminate stubs and normalize the presentation of the data. Rears have no slots; there is no purpose to having a distinct "limb stub" for them. It is better to transfer the only property they do have, armor, into the parent limb, than it is to try and maintain two different parsing trees depending on whether we're talking about rear or forward limbs.

I have a suggestion...

Please post away! That's what this beta-phase is for: to get your feedback to make these files better before we officially launch them.

[Noesis]

The parsing of existing XML for use from the game pak files into alternative data use is not difficult and being performed by player devs atm. The idea of remotely hosting up to date files by PGI however that is effectively managed and maintiaining a history of details also that can be accessed with confidence will help to automate the process more than having to manipulate local files. So switching from XML parsing to JSON will be a helpful step but nothing new to existing capabilities though the choice of JSON is a helpful step for the majority of web based applications or language use and supported frameworks, so thank you with that selection.

Longer term interests for the more applied use of API would also include actual pilot related data. This encompassing the existing stats we see as represented on the forum but also a significant amount of data yet to be capable for analysis. Items including information relating to drop data that players participate in and other data associated to a pilot not already included, like actual assests as opposed to use and other items not covered by the stats display like achievements, mech and piloting skills.

The additional aspect of personal pilot data however would suggest the need for security provisions and access to be controlled by the players to decide how and what data concerning them are applied. Some of this could be considered as per organisational representation within CW and player units, whereby certain pilot records could be provided if associated with a player unit. Interestingly of course CW will also evolve new data associated to units that will also ideally have a purpose for API use also, this in addition to perhaps appropriate Galaxy, political and economic information.

The ideal use of API is however more perceived as extending the use of applied data like player stats, drop information and other pertinent data in addition to the raw definitions associated with the mechs, tech and other useful reference material like map data etc. This is where player developers can then provide significant player and organisational tools that will help with the administration of groups and/or awareness to pilot performance and link it with any 3rd party tools developed for their own internal organisation purposes, as an example linking to tools associated with training provision, awards but more so to allow for the interpretation of strategical data to be able to make more objective and informed decisions.

The Templars are in a process of developing such infrastructure at the moment utilising differing parsing techniques based on player stats and the XML data. And then utilising this in combination with other functional needs aswell as reporting on the information for various strategical purposes this in addition to other various internally developed admin tools. The idea then of using more secure and controlled data as a hosted resourse that can be accessed with confidence but with appropriate security will significantly help 3rd party devs to achieve these aims, especially if drop data and other information relating to players and units associated with CW needs in the future would also be included for analysis. This is where the main interest of API provision would make the most significant difference with actual applied data analysis than simply looking at generic gaming definitions and would then provide much more significant data use with any API system.

Edited by Noesis, 13 July 2014 - 08:03 PM.

[Jon Cunningham]

Noesis, on 13 July 2014 - 06:24 PM, said:

Longer term interests for the more applied use of API would also include actual pilot related data.

Absolutely, once we nail down the static stuff, we'll be looking at that kind of functionality.

[Muzakman]

really happy to start seeing this stuff! Gonna take a bit to ponder and reflect on this, but my initial thought is that I'm loving the REST-ful way of addressing chassis (e.g., the /kf-s type of endpoints)

[Paikanx]

Jon Cunningham, on 14 July 2014 - 08:51 AM, said:

Absolutely, once we nail down the static stuff, we'll be looking at that kind of functionality.

Can't wait for that! We could make some great graph for me and my team!

[phatbhuda]

Yay for a API. I keep almost finishing a script and page scraping tool to collect my stats over time instead of just the cumulative view that the site offers.

[Toadflakz]

A comment on the CW mapdata.json file - I would have expected you to use JSON array syntax for the planets rather than having them as individual properties. This has meant I've had to write a custom JSON parser for this data file in order to transform it into an array of data.

However, this is my first foray into working with JSON - am I unrealistic about this being an issue?

Edited by Toadflakz, 22 December 2014 - 01:29 AM.

[Reno Blade]

phatbhuda, on 12 December 2014 - 11:47 AM, said:

Yay for a API. I keep almost finishing a script and page scraping tool to collect my stats over time instead of just the cumulative view that the site offers.

I think someone made a tool to collect data already.
Was names "improved statistics" or something.
I think its this post:
http://mwomercs.com/...ved-statistics/
EDIT: it seems he discontinued the program...

Edited by Reno Blade, 22 December 2014 - 03:22 AM.

[Noesis]

Reno Blade, on 22 December 2014 - 03:19 AM, said:

I think someone made a tool to collect data already.
Was names "improved statistics" or something.
I think its this post:
http://mwomercs.com/...ved-statistics/
EDIT: it seems he discontinued the program...

Quote

Yes, PGI broke the statpage, no mechnames there.
Did anyone count how often they broke the page? Must be every second mech they introduced.
Sorry, but I cannot support this tool any longer with having to clean up after them so often.

[Nadezhda]

Hi, can the next MWO client include a web browser under the Faction/Unit tab?

The default browser page can be directed by the unit leader at the unit management tab, or point to a generic one by MWO.

That way, units can design their javascript/json based websites that is accessible from in-game clients, which could allow unit members to chat to one another using their in-game personalities (API), and can leave each other messages for the next time members log on or off, among other things.

Many players (especially older ones) are not very familiar with alt-tab and windowed fullscreen and downloading 3rd party software (eg Teamspeak). If it can be done from within the game client it would be much better for everybody. And if Steam is an ambition then you might want some help to get these things organized.

And please make this browser accessible while queuing for a match.

[Appogee]

Has any progress been made on enabling us to access pilot-specific data?

[Kiiyor]

Jon Cunningham, on 11 July 2014 - 04:27 PM, said:

Please post away! That's what this beta-phase is for: to get your feedback to make these files better before we officially launch them.

Hi Jon, any progress on a pilot specific API? I'd love to get my greedy little mitts on it.

Edit: So, I can add 'Jon being gone' to my list of things I didn't know about.

Edited by Kiiyor, 10 September 2015 - 10:05 PM.

[Vetal]

Does it work? Seems mech data isn't available because of 500 error. I'm developing mobile analog of smufry labs, and this is necessary for me to build data structure in application. I'd like to download it to make application work both online and offline.

Edited by Vetal, 13 September 2015 - 10:20 PM.

[Vetal]

I'd like to say that it doesn't matter what kind of source (xml or json), I just need to have it in order to code the tool, and make it capable to update as soon as possible when the game is updated.

Edited by Vetal, 15 September 2015 - 09:22 PM.