305 replies to this topic

[w0qj]

Guys, let's not be so hard on MWO...

1. IF MWO programmers had the foresight and resources to program for contingencies 10+ years into the future, it would be called

Over-engineered

ie: cost of this would be prohibitive, cost=high means less F2P players, causing cost to go even higher, to the point that MWO does not make any money, and servers are shutdown as a business decision, and well before the 10+ year mark.

Many online games don't last over 3 years, so why should MWO plan for 10+ years into the future?

2. What we are seeing is there's been a dramatic rise in number of mechs in the last 2-3 years, and the increasing number of bolt-ons were compounded by this rise in number of mechs as well.

(a) What is the answer for 1x2x3x4x5x6x7x8x9x10 = ?

(b ) What is the answer for 1x2x3x4x5x6x7x8x9x10x11x12x13x14x15 = ?

The difference in magnitude between answers to (a) vs (b ) may surprise you.

I believe this is one of the factor leading up to this approaching/exceeding maximum database count issue, thus causing this great server instability.

(That said, I've succeeded in leveling up a number of my mechs, got my WHK free mech for Nov, etc.)

Edited by w0qj, 03 December 2023 - 12:39 AM.

[Matt Newman]

Tarl Cabot, on 03 December 2023 - 12:12 AM, said:

Matt,
Back tracking on my previous post, thus asking for clarification. Thus a question on the no server found. Is just that player affected or all 24 players for that actual server instance? Just from the lack of PSR movement, it would appear to be that server instance.

The entire match fails. No one is in the match. No PSR changes as the match does not officially spin up.
Unfortunately the Mechs still get locked as that happens when the match group is created, the map and mode are chosen, and all the information is sent to the server.

PS going to bed now

martian, on 03 December 2023 - 12:08 AM, said:

Maybe with the company, but not working on MWO - at least that is how I understood the Russ Bullock's words when he described the current MWO status. Do I get it right?

Ah I see what you mean. Yes on other projects but I can always call on them in times such as these!

[martian]

Matt Newman, on 03 December 2023 - 12:42 AM, said:

Ah I see what you mean. Yes on other projects but I can always call on them in times such as these!

That is good to know.

[Besh]

Matt Newman, on 03 December 2023 - 12:33 AM, said:

Those are fair questions and I have to admit I am not an expert! The presenting behaviors in the software do feel like memory overruns but I am assured by people way smarter than I am that it is the overall number of items in the database that is causing these issues. The Intermittence AFAIK can be chalked up to large data transmission?... lets pretend our database is a Tweet on twitter and we have been keeping every tweet under the Character limit of 300 but somehow we just start tweeting at 310 characters and Elon Musk is mad at us!? (I am getting further out of my knowledge comfort zone must switch back to food based metaphor!!)

Also I am glad you like my Kitchen Metaphor I will say that yes as humans we don't have to load every ingredient when we are making a specific dish! BUT think about how many dishes you know how to make! Or how you know what spices and flavors to add to get a specific flavor its almost like you DO have a Database in there right?

Well, IT in principle is/was modelled after how it is/was understood at times how brain/mind works .

Looking at some items involved in creating the Issue...

ItemDatabase : Assumed it has an index limit, this limit is ( should have been ) known and observed at all times for obvious reasons;
Data transfer : it seems transferring the entire ItemDB for each Match isn't relly efficient ;
Load: for both, Database and MatchInstances, load is (probably unnecessarily) high when having to transfer/load the entire ItemDB for each Match .
HW considerations : each instance having to load the entire ItemDB does not lead to unnecessary bloat in HW requirements, making the System more vulnerable to simple HW failures ?

In short : Interesting ! Designing this Game 12 years ago, one would do it with the prospect of growth in mind . Growth for MW:O inevitably means "More Matches" ( amongst other things ) . So one aspect I would go ahead and make as resource efficient as possible is how the System handles increasing number of Matches . But (honestly) what do I know ? Ofc hindsight is 20/20, and I am basically somewhat trying to reverse engineer the thought processes from 12 years ago without knowing anything about the constraints an challenges that had to be dealt with . Which obviously is an endeavour stretching the scope of this Thread a bit .


Anyhow . Thanks again for spending time with us at the weekend .

Sleep well .

Edited by Besh, 03 December 2023 - 01:19 AM.

[Besh]

w0qj, on 03 December 2023 - 12:35 AM, said:

Guys, let's not be so hard on MWO...

1. IF MWO programmers had the foresight and resources to program for contingencies 10+ years into the future, it would be called

Over-engineered

ie: cost of this would be prohibitive, cost=high means less F2P players, causing cost to go even higher, to the point that MWO does not make any money, and servers are shutdown as a business decision, and well before the 10+ year mark.

Many online games don't last over 3 years, so why should MWO plan for 10+ years into the future?

2. What we are seeing is there's been a dramatic rise in number of mechs in the last 2-3 years, and the increasing number of bolt-ons were compounded by this rise in number of mechs as well.

(a) What is the answer for 1x2x3x4x5x6x7x8x9x10 = ?

(b ) What is the answer for 1x2x3x4x5x6x7x8x9x10x11x12x13x14x15 = ?

The difference in magnitude between answers to (a) vs (b ) may surprise you.

I believe this is one of the factor leading up to this approaching/exceeding maximum database count issue, thus causing this great server instability.

(That said, I've succeeded in leveling up a number of my mechs, got my WHK free mech for Nov, etc.)

Exploding Database due to constantly adding Items at sometimes rapid pace can be a challenge in itself I guess .
Having to load that entire DataBase for each Match individiually in context of a growing online teambased PvP shooter sounds like the kind of nightmare I would not wish onto enemies .

p.s.: I am not trying to be hard. I am trying to understand. I realize language/culture differences may lead to misunderstandings . Like, me being german, I am very much used to putting stuff in " " when quoting, and I find myself doing this a lot on forums/boards instead of hitting the quote button then editing, since its just faster - esp. when quoting phrases instead of an entire paragraph . But me quoting what someone wrote in " " may make some think I do it because I doubt what was written .

Edited by Besh, 03 December 2023 - 01:18 AM.

[Hagen Zwosta]

Are the servers completely gone now? 4 locked mechs in a row and counting.
edit: suddenly its fine.

Edited by Hagen Zwosta, 03 December 2023 - 02:11 AM.

[Steel Mike Eels]

So I have a question:

Is this an issue with the actual database software in use (ie whatever is being used cannot handle that many records?) or is this an issue with the throughput of raw data when matches are queued up?

[tenchugecko]

Dear Guys

You should be very hard to "MWO" about this fuckup.

I any IT Project, you will calculate the sizes and frequencies of data, and find out maximum Values Systems/Servers/Clients can handle

Instead they just misused the item system in a hack for an insignificant feature like boltons.
It's not OverEngineering, it's just plain 1st semester informatics to know not to do that.
And just like the GTA Item Cross Reference bug, this **** was smelling from afar -
with MWO Startup Times scaling negatively with avaiable computing power through the years.

All of this, while having no option to disable bolt-ons temporarily.

But what do i know as a Software Project Manager.

(PS. 3 times disco today.)

[Viltor]

Do I understand this right?
Game is broken, and devs don't know how to fix it?

[Besh]

Steel Mike Eels, on 03 December 2023 - 03:22 AM, said:

So I have a question:

Is this an issue with the actual database software in use (ie whatever is being used cannot handle that many records?) or is this an issue with the throughput of raw data when matches are queued up?

...or maybe a memory allocation issue, i.e. db *suddenly* having become too big for individual ones OR the total # of MatchInstances simultaneously being run on the available resources, as it seems like every instance has to load the entire db . If that, could be HW/SW/combination .

A "simply indexing" ( not sure I understand what ItemDB for MW:O does, and how, hence " " ) DB *should* be expandable .

Edited by Besh, 03 December 2023 - 06:20 AM.

[ExoForce]

Long time ago somebody posted here somewhere the detailed comparison between cryengine and other game engines in terms of handling in-game objects. IIRC in simple words it was cryengine (as implemented in this game) is not good for the hit registration of multiple sub-objects on main item.

Elders are remembering what clan Lrms and MGs did to servers.
IIRC the internal hot-fix was that servers where dropping half of the damage (or not registering half of the shoots).
Official explanation (after quite some time) was like *server hitreg timer was wrongly configured*...

Praise the cryengine (as implemented in this game).

Edited by ExoForce, 03 December 2023 - 07:11 AM.

[martian]

Viltor, on 03 December 2023 - 06:06 AM, said:

Do I understand this right?
Game is broken, and devs don't know how to fix it?

No, you do not understand it right.

Read this update, please: Mwo Status Update And Known Issues - Update 2 - Dec 1St

[Saved By The Bell]

Use oceanic server, while its still somehow alive.

Dont blame the PGI team, because its not help anyone.

[ScrapIron Prime]

Viltor, on 03 December 2023 - 06:06 AM, said:

Do I understand this right?
Game is broken, and devs don't know how to fix it?

Incorrect. They know how to fix it, the issue is the hard choices required to do so.

The core engine uses a database circa 2003. Older databases had a much lower maximum record count than databases today, so the task before them is to reduce the record count in the database by enough to not just support everything in the game now, but leave extra space for everything they might add to the game in a couple years.

they’ve chosen to reclassify and combine bolt ons so that multiple kinds can use a single database entry, rather than simply deleting content. This is not simple, but they know what has to be done. Expect certain bolt ins to be goofed up for a while as they work out the fine details.

the other option would be to upgrade the database engine of the game… but that’s making a whole new game engine…and that’s totally out of scope.

Devs… as a guy who’s been a computer engineer for 32 years, I’ve been where you are now. You’ve got this.

[Thunderbuddy]

As a brand new player with 1 mech, it is very offputting to have my mech locked and not be able to locate matches. This game is way too old to be having open beta level issues.

[w0qj]

A warm welcome to MWO!

Here's how to get your free 2nd mech, if you input this redeem code immediately (time-limited offer!):

Redeem Code:
CS23-EQV5JDX

Use above code after logging into your account: https://mwomercs.com/profile/redeem

- - - - - - - - - - -
Source:
https://mwomercs.com...-mech-from-cs23

Thunderbuddy, on 03 December 2023 - 10:05 AM, said:

As a brand new player with 1 mech, it is very offputting to have my mech locked and not be able to locate matches. This game is way too old to be having open beta level issues.

[martian]

Thunderbuddy, on 03 December 2023 - 10:05 AM, said:

As a brand new player with 1 mech, it is very offputting to have my mech locked and not be able to locate matches. This game is way too old to be having open beta level issues.

Read this explanation, please: Mwo Status Update And Known Issues - Update 2 - Dec 1St

The situation should be resolved in ten days.

[CrustyMech]

Besh, on 02 December 2023 - 11:15 PM, said:

But why does it load the entire Database at match/lobbycreation ? Why does it not just pull whats needed for the specific 24 'Mechs/loadouts/BoltOns etc. neeed for that specific Match ?

It could be a limitation on the engine.... it might be the way the game was originally coded and the complexity of modifying the code is not feasible.... or something else . I spent 40 years in IT including working with huge/complex database systems for our local health care system. There was NEVER a quick fix...

Edited by CrustyMech, 03 December 2023 - 10:35 AM.

[Maxx31]

What about any Premium Time that was burned due to the issues? I would say I agree there is not a 100% allocation (games that do run, sleep, eating, etc).

[Strelok7]

Besh, on 02 December 2023 - 11:15 PM, said:

But why does it load the entire Database at match/lobbycreation ? Why does it not just pull whats needed for the specific 24 'Mechs/loadouts/BoltOns etc. neeed for that specific Match ?

I feel like a "very loaded answer" today - here you go.
Binary logic vs human logic (*although humans do use it and fall for it in political populism...for example).
I think there is you answer - pull > pull from what > database > where is the database > lets load it > operation success
why is it not always loaded? > server resources should not be always loaded (waste) RAM vs HDD
Human brain example - we don't need to walk around all day, thinking about making a burger, when we should be fixing our car. That is ADD. Usually caused by Stress and Anxiety. *Rest is the solution. Straight functioning brain keeps priorities strait. Just in time information.

Matt Newman, on 03 December 2023 - 12:33 AM, said:

Also I am glad you like my Kitchen Metaphor I will say that yes as humans we don't have to load every ingredient when we are making a specific dish! BUT think about how many dishes you know how to make! Or how you know what spices and flavors to add to get a specific flavor its almost like you DO have a Database in there right?

Brilliant metaphor I was conveying your metaphor to someone who is not too interested in this stuff this morning - they loved it too!