dMZX Forums: Higher Maximum String Size / User-Definable Max String Size -> Archived Requests -> Tracker

Okay, I tried to get Lancer to post this like a bazillion times and he didn't, but here's what we were talking about before.

#1. & (I presume, but he didn't name a specific game) uses strings much longer than 1MB by exploiting INC $string.

#2. Since MZX now enforces MAX_STRING_LEN in inc_string, Lancer-X estimated that the required MAX_STRING_LEN to properly run his game would be around 16MB. (given that Lancer's games have worked on pretty much every system up until now I don't see a real problem here)

#3. Lancer-X also requested a per-world counter to allow the user to define the maximum string length themselves in the future (min. ver. 2.85).

With #3 we'd probably see some people doing "cute" things like set "MAX_STRING_LENGTH" "(-1>>1)" / set "$str.(-1>>1)" 0, but then again if they really wanted to they could create a couple of thousand maxed out strings right now and have the same effect. I was also thinking of something like a platform-definable cap on MAX_STRING_LENGTH that fires off warnings when the game tries to set the max higher than that.

& is probably fine with the 1mb limit, I meant more generally. I just feel there's legitimate reason to use larger strings and I know I've used them before.

1) Sounds like a compatibility check could remove the limit check for old games. This by itself doesn't justify changing anything else.

2) Desktop systems, sure. You can rule out running such games on all the console platforms, though. The NDS port would have a heart attack if you asked even for 1MB in a single realloc() (but that platform is comically limited already).

The point of the limit was to catch broken games that would de-reference a string at a buggy offset. MZX responds by realloc()'ing the string, even though the interluding bytes aren't used. That's a recipe for OOM in test mode.

IMHO having a limit and many strings encourages a better overall design than having single strings with no size limit. It makes people think about what they are doing and how it might be wrong to need such a large working set of data. Also what you said about them being equivalent isn't really true -- some platforms have tiered memory which will handle multiple smaller virtual memory objects better (NDS).

I think this setting should be a world/board property rather than Yet Another Weird Counter. If you're going to allow the limit to be changed at runtime you might as well just ditch the limit and suffer the consequences (if any).

--ajs.

Also, to speak to what Lancer was saying, more generally..

There are plenty of limits in MZX people have asked to have increased/changed over the years. In the past we've said in MZX3/GCS-next we'll make that change but we've tried to retain some of MZX's "style" by not engineering around certain limits. It just seems weird to me that games are limited to 16 color palettes (yes yes, SMZX) but they can allocate 2GB strings.

If we do change the limit policy, I'd vote strongly to keep a limit, and to make that limit reasonable (16MB still seems like too much to me). Making the limit "configurable" (and thus unlimited) doesn't really seem appropriate for something that is supposed to be a sandbox. It might even be argued that games shouldn't be able to create an unlimited number of strings, maxing out system memory, or that MZX should have a way to report OOMs back to the game engine so it can adapt.

--ajs.

Cases like the NDS is why I mentioned a per-platform hard limit, and I never really liked the idea of #3 myself.

Okay, 16mb might be too much, but 1mb is too small. 4mb?

4MB fixed limit seems OK to me.

BTW, I made some references to NDS, but you're right to ignore the console platforms here because most of the ports _can't_ run modern, "high end" games because they just don't have the memory/processor power to do it. I'm was never really interested in "lowest common denominator" type engineering either. That's why the string limit was already a generous (?) 1MB.

EDIT: Totally missed this but I had to comment on it:

Quote

In principle (see below for reality) I think people expect to run all the same games on the ports. It's kind of moot if a game or MZX itself sets a different limit on a given platform. The game isn't going to know it needs to work differently unless it can read the max string length, and even if it could, it's starting to get difficult to pull off. People aren't going to do it. I think it's better that the game goes OOM in that case than simply behaves incorrectly because its strings get truncated differently on one platform than on another.

Going OOM in the ports is familiar for anybody who's using them -- kvance told me the biggest problem he had on NDS was memory consumption. The old 2.51 games that relied on all the old 64k segment limits would run acceptably, especially after he added extmem paging, & I put in a lot of work to cut down the size of MZX itself to claim back some more memory for games. But we never really expected new games to work; almost every feature added since the port has increased MZX's base requirements beyond what the first gen handheld consoles can handle. Strings are a major offender, obviously.

--ajs.

This post has been edited by ajs: 30 June 2012 - 11:33 PM

Updating status to: Awaiting Feedback

Limit increased to 4 MB in GIT 187311d.

Updating status to: Implemented
Issue fixed in: GIT (debytecode)

Well, since the limit was increased ages ago and the user definable string size didn't go anywhere, I guess I'll just

Moving to: Archived Requests