26 replies to this topic

[Jon Cunningham]

Hello folks -

I wanted to let you know we will be coming out sometime in the future with JSON files on the CDN to describe mechs, parts, and equipment. Prices from the store included.

Items
http://static.mwomer.../list/full.json

Omniparts
http://static.mwomer.../list/full.json

Mechs
http://static.mwomer.../list/full.json
http://static.mwomer.../list/dict.json
http://static.mwomer...view/kfx-s.json
(or any mech code)


These files are just an advanced sneak peak, they’re not final and are subject to a great deal of revision before we release them from Beta. They probably contain a couple bugs. We’re not certain when we’ll launch them, but we did want to make sure folks who are working on third party tools were aware we’re working on them and can offer some feedback if they like.

Let me know if anything catches your eye or you have any requests.

[Surtosi]

I'm not a dev by any means, but i do like that there are life support and sensor modules. Love to see effects from those things getting knocked out or disrupted.

[Muzakman]

I've been playing around with integrating them into the MWO Command Console app, love the work that's been put it on this.

Here's some notes so far from using it
EDIT: clarify thoughts over course of afternoon and getting more time in on it

• I like the /mechs/list/dict.json, but from a RESTful (and semantic to me) perspective, /mechs/index.json makes more sense.
  
• I like the use of a natural (mech or part name) key to index the mech/view/(name), but it really feels that it should be /mechs/views/(MechID) - /mechs/views/117. From the model standpoint, it's a bit odd that the object graphs are indexed by name rather than ID in some places and by ID in others.
  
• Please make the "Name" property field be a UI display name or provide a separate display name property. MediumLaser forces me to either look up a display name elsewhere or try to compute it (will work, but is brittle).
  
• Perhaps API clients could specify the appropriate ACCEPT HTTP headers to negotiate content language. en-us could be default if not specified / invalid headers / unknown language
  
• I'd prefer if property names could be consistently cased. Some are capital, some are lower -case e.g.:

  "Health": 18,
  "slots": 10,
  

Edited by Muzakman, 21 July 2014 - 02:36 PM.

[Jon Cunningham]

Thanks for the feedback. The inconsistency of capitalization bothers me as well.

[Muzakman]

EDIT: Happy to contribute, hope my feedback is helpful!
I feel that the capitalization and spacing conventions used in the Smurfy API would be ideal, since I'm used to seeing

mech_id

instead of

MechID

, same with

family 

rather than

Class 

, etc...

Oh, speaking of Class, forgot to mention a bug I found:

when accessing

/api/mechs/list/full.json

, every 'mech has Class: "Light", regardless of actual Class.

Another property request:
can you provide a Hardpoints aggregate on the Mech object that provides a total hardpoint counts by category (AMS, ECM, Ballistic, Energy, Missile)? I'm doing it now when mapping the API data into my app's Domain, here's some JS (underscore lib is really handy)

EDIT: forgot to mention why I ask for this: the aggregation is tricky working with OmniMechs, and can be brittle.

m.hardpoints = _.reduce(_.pluck(_.values(m.Loadout), 'Hardpoints'), function (memo, next) {
						memo.Ballistic += next.Ballistic;
						memo.Energy += next.Energy;
						memo.Missile += next.Missile;
						memo.AMS += next.AMS;
						memo.ECM += next.ECM;
						return memo;
					}, {
						Ballistic: 0,
						Energy: 0,
						Missile: 0,
						AMS: 0,
						ECM: 0
					});

Edited by Muzakman, 22 July 2014 - 01:23 PM.

[Jon Cunningham]

Thanks for the feedback, Muzakman! I'll update with a fix to the bug you mention.

I've been leaning toward and away from what naming convention to use, but based on your feedback I will probably go for all lowercase.

[Jester McCloud]

any words on getting user stats, etc? I've been playing around with scraping the profile stats page, but, I just kinda hate web scraping..... Too fragile.

[Jon Cunningham]

Either in the next API update or the one after that we'll be looking at it. : )

[WVAnonymous]

Official json user stats would be great.

Any thoughts on whether or not the complete history (pre-stats reset, back to closed beta, from closed beta) would be available?

[Jester McCloud]

Yes, the reason I want user stats is to track stats in more detail. For instance, changing a load-out and track the stats for a month or so, then change the load-out and track that.

It would be too much to ask the mwo server to do this, but individual computers could.....

I also play Eve Online. It is nice that not only do they have a great API, but when joining a Corp, you can provide a token that lets them see and download interesting things. That might come in handy with CW.

[Flying Fox 333]

Jon Cunningham, on 30 July 2014 - 11:19 AM, said:

Thanks for the feedback, Muzakman! I'll update with a fix to the bug you mention.

I've been leaning toward and away from what naming convention to use, but based on your feedback I will probably go for all lowercase.

I personally don't mind upper or lowercase myself. Am adaptable. As long as the API itself is maintainable for you at PGI too.

I am currently working on a statistic collection, analysis, reporting and submission application to go with MWO and want to eventually release this. It relies heavily on reading the profile statistics and end match data, neither are ideal as the web site design changes and the end match screen can be misinterpreted.

Having said that I am curious as to what sort of update procedure you will have as new battlemechs, weapons and so forth are added to the game. Will this update to the API data be part of the fortnightly update/maintenance cycle? Have you considered in your designs additional objects like the NPC/CW content (turrets, dropships, vehicles and so on)?

Since Muzakman found some errors in the data, will the data from the API in future be derived from a validated source? I am just concerned about using the API as the sole source of data if it doesn't match what's in the game files as well.

Naturally as people don't (and shouldn't have to) provide their login details to use something like this. Are you planning on using the User ID thats in our profile page to allow this to be used in lieu of the username/password to gather information from the API? A Group/Corp ID could also be used for group based data as well.

I am interested in also knowing how you will handle the volume of requests to the dynamic data. For example as part of my design I am intending to provide a custom user-agent header with my requests so that it assists PGI in tracking with regard to 3rd party applications using the API. Has any requirements been established so far about what the 3rd party application needs to provide beforehand? Do we need to apply for a key that is in turn used with the User ID to determine the application and user requesting the information?

Finally thanks Jon for being so open and happy to take feedback. I've not heard of a game actually asking for feedback during development of its API before, so am quite excited about this.

[Jon Cunningham]

Flying Fox 333, on 01 August 2014 - 06:18 PM, said:

I am currently working on a statistic collection, analysis, reporting and submission application to go with MWO and want to eventually release this. It relies heavily on reading the profile statistics and end match data, neither are ideal as the web site design changes and the end match screen can be misinterpreted.

That sounds pretty cool. Good luck with development.

Quote

Having said that I am curious as to what sort of update procedure you will have as new battlemechs, weapons and so forth are added to the game. Will this update to the API data be part of the fortnightly update/maintenance cycle?

Yes, changes are for the most part automatic and new content is added without any additional work on our part.

Ie:
static.mwomercs.com/api/mechs/view/kto-18c.json

Quote

Since Muzakman found some errors in the data, will the data from the API in future be derived from a validated source? I am just concerned about using the API as the sole source of data if it doesn't match what's in the game files as well.

Eventually, yes. That's why we chose to go with an open, extended beta. We'd rather make a good product that meets people's needs than just "officially announce" a brand-new but-mostly-broken api out of the blue. It's also why we're doing the beta quietly. There'll be a big post when it is officially released.

Quote

Naturally as people don't (and shouldn't have to) provide their login details to use something like this. Are you planning on using the User ID thats in our profile page to allow this to be used in lieu of the username/password to gather information from the API? A Group/Corp ID could also be used for group based data as well.

Currently, our plan is opt-in (users can choose to allow their data to be accessed).

Quote

I am interested in also knowing how you will handle the volume of requests to the dynamic data.

We'll want to break the data into highly-dynamic and merely-dynamic, so that our CDN can serve a JSON file with permitted user details for things that change, but aren't urgent (number of deaths / kills). When it comes to highly-dynamic or potentially anything like allowing write access, it's almost certainly going to be locked by API key. Our security model for that isn't flushed out yet. Suggestions are welcome.

Quote

Finally thanks Jon for being so open and happy to take feedback. I've not heard of a game actually asking for feedback during development of its API before, so am quite excited about this.

Thanks! Hope it's useful for you.

[Fire and Salt]

Jester McCloud, on 30 July 2014 - 05:47 PM, said:

any words on getting user stats, etc? I've been playing around with scraping the profile stats page, but, I just kinda hate web scraping..... Too fragile.

Heh, thought I was the only one. Guess I should have found this forum sooner...

Scraping a players mechs is easy, from this page (I planned to let the player use a tool to do this themselves, since it requires login):
https://mwomercs.com/profile

I was hoping to calculate if the mech was basic/elite/master by scraping this page:
https://mwomercs.com...stats?type=mech
but the XP earned doesn't add up right... I earned less XP than is required to master my stormcrows, but they are all mastered in game... maybe premium time throws it off? I know using GXP or converting to GXP could throw it off, but I didn't do that with any of my stormcrows...

[Escobar]

Nope, not the only one.

If you lean more towards .NET, I can provide the class library I use for mechStats to 1) login 2) scrape stats form pages 3) scrape additional info (whats in your mech bay, your titles, etc...). I have several unit test cases that I run regularly to ake sure that some web design change hasnt broken anything.

clarifying: by 'class library' I dont mean 'here is this 3rd party scraping package' or anything like that. I meant, a strongly typed scrape of MWO specific stuff:

• public class MWOPilotStatistics
  
  • public BaseStatistics GetBaseStatistics()
    
  • public System.Collections.Generic.List<GameModeStatistics> GetGameModeStatistics()
    
  • public System.Collections.Generic.List<MapStatistics> GetMapStatistics()

and so on, with accompanying:

/// <summary>
/// Base pilot statistics
/// </summary>
public class BaseStatistics
{
public long CBills { get; set; }

public long Deaths { get; set; }

public long Kills { get; set; }

public long Losses { get; set; }

public long Wins { get; set; }

public long XP { get; set; }
}

and the rest

If there are any other .NET development efforts going on, id be happy to seed this on sourceforge or something

Edited by Escobar, 11 August 2014 - 08:09 AM.

[Escobar]

I had been doing some updates myself based on the new static data available, and so went ahead and created a codeplex site for .NET related development for user code. I have seeded it with my MWOSiteReader project for scraping strongly typed data and the initial API for 'items' (with its initial focus being on weapons, their normal damage, and their statistical critical 'Max/Best Damage').

Once the publicly available player stats comes, ill start bringing over most of mechStats in to codeplex as open source.

If you find any use of it, or if it shaves twenty minutes off your dev work, knock yourself out, and give me a shout if you see me in-game (note, that's 'shout' and not 'shot' - I get plenty of those already )

MWO User Code on CodePlex

[MasterBLB]

These files looks good,very detailed.Most fields are self-explanatory,however some documentation would be nice for ex:
AC20:
- minheatpenaltylevel (ah,that's number of weapons of the type required to trigger ghost penalty.However is was unclear when I 1st saw it,I figured out what's that when I checked medium laser)
- lifetime
- maxdepth

LRM20:
- heatPenaltyID
- peakdist
- peaktime
- trackingstrength (I guess it describes how long lock is maintained when the target goes out of field of view,but I'm unsure)
- radius (that's for splash damage?)
- spread

Too bad JSON does not supports comments :/
So,as I understand the work is in progress anyway maybe it'd be better to switch to XML standard?What you say John?

Besides,at the very top it'd be useful to add:

"lastupdated"  : DD:MM:YYYY HH:MM

that'd help to determine if any stuff based on your JSONs is using most recent versions.
Perhaps a list of changes would be nice too:

"changelist" :
{
	"PPC" :
	{
        "projectilespeed" : 1500
	},
	"Clan ER Large Laser" :
	{
        "beamduration" : 1.5
	}
}

(not sure if above syntax is JSON-correct,but I'm C++ guy and I have java-style parentheses.Besites,it's for demonstration purposes only)
assuming I've written an offline mechlab tool having such change list I could inform an user when he/she opens a loadout what exactly has been changed.

And the last thing,placement of these files.If I may advice something,the best would be locally in <MWO folder>\Game\JSON ,along with some documentation.txt/pdf at least these mech/equipment related ones.

Hope that was helpful,
MasterBLB

Edited by MasterBLB, 15 August 2014 - 02:54 AM.

[Noesis]

JSON provides a more readily applied medium for internet use as it is readily convertable into other html family language use like PHP. Considering that a number of units may wish to apply tactical and strategical data on their forums or sites there is a need for readily available conversion proccesses to be applied for this need. This then linking and helping any admin processes already set up.

However it is not that much more complicated to utilise a XML conversion process either, but JSON at least allows for a readily convertable access of the data into PHP arrays that simplify the process.

For stand alone applications then that may prefer to use XML text over JSON text then perhaps it would be best to petition to have both formats as an output as opposed to replacing the existing usesful JSON use?

---

For static data (weapons, mechs, etc) the idea of local files might be useful if readily updated as a part of each patch as applicable. But if the idea is to later expand to an API that use more dynamic data relating to pilots, drops, economy, CW info etc. Then some remote access to server driven output that updates on a more frequent basis with the appropriate security protocols to these kinds of data provisions about individual pilots than the use of local files will be a much better medium to use.

Edited by Noesis, 15 August 2014 - 05:01 AM.

[evilC]

MasterBLB, on 15 August 2014 - 02:50 AM, said:

- trackingstrength (I guess it describes how long lock is maintained when the target goes out of field of view,but I'm unsure)

Tracking Strength is the maneuverability of the missiles (ie turn rate deg/sec)
Artemis increases tracking strength.

[evilC]

Having a database background, I am not so keen on the horrendous repetition of data.

Can you not normalize at all?

For example, with the Kitfox JSON file, why are all the properties of the components included?

For example:

					"name": "C-MACHINE GUN",
					"id": 1209,
					"equiptype": "weapon",
					"category": "weapon",
					"removable": true,
					"factions": {
						"Clan": true,
						"InnerSphere": false
					},
					"description": "A rapid fire, high caliber, automatic gun system.  This weapon is most useful when targeting internal structure after armor has been removed from a component.",
					"stats": {
						"damage": 0,
						"heatdamage": 0,
						"heat": 0,
						"cooldown": 0,
						"ammoType": "ClanMachineGunAmmo",
						"ammoPerShot": 1,
						"minRange": 0,
						"longRange": 120,
						"maxRange": 240,
						"tons": 0.25,
						"slots": 1,
						"speed": 100
					},
					"url": "http:\/\/static.mwomercs.com\/data\/weapons\/1209.json"
[color=#000000]},[/color]
[...]
				{
					"name": "C-MACHINE GUN",
					"id": 1209,
					"equiptype": "weapon",
					"category": "weapon",
					"removable": true,
					"factions": {
						"Clan": true,
						"InnerSphere": false
					},
					"description": "A rapid fire, high caliber, automatic gun system.  This weapon is most useful when targeting internal structure after armor has been removed from a component.",
					"stats": {
						"damage": 0,
						"heatdamage": 0,
						"heat": 0,
						"cooldown": 0,
						"ammoType": "ClanMachineGunAmmo",
						"ammoPerShot": 1,
						"minRange": 0,
						"longRange": 120,
						"maxRange": 240,
						"tons": 0.25,
						"slots": 1,
						"speed": 100
					},
					"url": "http:\/\/static.mwomercs.com\/data\/weapons\/1209.json"
				},

Storing the stats of a laser within the mech JSON is bad practise. If you change one of the properties of the weapon, this file becomes invalid.

Instead you should just refer to the unique key for the weapon, and require further requests to retrieve that information.

So the above should instead be:

				{
					"name": "C-MACHINE GUN",
					"id": 1209,
				},
 
[...]
 
				{
					"name": "C-MACHINE GUN",
					"id": 1209,
				},
 

or even just:

				{
					"id": 1209
				},
 
[...]
 
				{
					"id": 1209
				},

Edited by evilC, 15 August 2014 - 05:17 AM.

[Escobar]

MasterBLB, on 15 August 2014 - 02:50 AM, said:

[color=#959595]assuming I've written an offline mechlab tool having such change list I could inform an user when he/she opens a loadout what exactly has been changed.[/color]

Id rather not see a changelist, since it would have to be more of a changelog, including changes from every version to the next, in case you last refreshed at v 100 and next updated at v 110. Better to store locally what you saw at v 100 and calculate the diff yourself when you update to v 110. And a great idea of presenting a notification of build performance changes based on when the build was created/updated and the latest weapon/mech changes.

MasterBLB, on 15 August 2014 - 02:50 AM, said:

Besides,at the very top it'd be useful to add:

"lastupdated"  : DD:MM:YYYY HH:MM

that'd help to determine if any stuff based on your JSONs is using most recent versions.

Yes, also would find that helpful. Although we mightaccomplish the same thing by using the patcher client file version information found in http://patcher.mwome...chInventory.txt
The last line of that text file is the client release #. It might work, although there may also be server side changes that do not require a client push, so then again it might not.

Edited by Escobar, 16 August 2014 - 05:12 AM.