[Exophase]

Nightwatch, on May 18 2008, 01:25 AM, said:

Ok, feedback time.

After fighting some more with the editor, I'm not convinced that the robotic editor is that good of a match for Lua. The Robotic editor is fundamentally tied to the concept of a line as bytecode, and Lua doesn't work that way. It's very difficult to do even simple things like commenting out code properly, and the number of bugs related to the robotic editor is bad enough without hacking a clumsy generic line editor into it. Not saying that Exo did a bad job, of course... I think the "Robotic is bytecode and its text representation doesn't really exist" design makes it difficult no matter what you do.

I agree 100%. For years I've been preaching that Robotic can't move forward so long as it's tied to its bytecode like this, and it has really held us back from improving the language. I had planned several times to do two things as an initial push to revising Robotic: make an extension allowing Robots to be stored as source as opposed to bytecode, and writing a highly general macro system for Robotic to allow it to be extended. Unfortunately the plans for the latter got out of hand and I don't really have the focus/interest to work on MZX anymore anyway. Nonetheless, the first part shouldn't be that difficult.

As you mentioned, the port was done really just to give MZX compatability first, with most of the (relatively minor) improvements focusing on improving the editing aspect; the internal representation in particular was left intact. Nevertheless, I don't think hacking the Robotic editor to be a general line editor is as bad as it sounds. The one I wrote already has the ability to use "unverified" lines, you can just go from here and remove the verification/compilation stage at the end (or replace with a Lua one), and store the line buffers sequentially as opposed to the compiled bytecode (and if you didn't know, you can comment/uncomment a line with ctrl + c)

As far as expanding the MZX format and editor in a simple way to support this, I would suggest just using a robot bytecode length of 0. I normally don't like hacks like this, but I'd hate to see the MZX world format changed in such a way that makes it difficult to maintain compatibility. I wouldn't go for version stringing it, there might be useful reasons to have mixed robots, at least at first (like for loading old MZMs)

Then it can go something like this: robot structure has bytecode and source buffers. Upon loading, if length is non-zero, disassembler is used to convert bytecode to source (this should be pretty straightforward), length is set to zero with whatever other header extension stuff placed in. You can either abandon the bytecode or not - I strongly recommend throwing it out because the current bytecode sucks and it'd be good to have something much better. Robot editor loads line buffers with source directly instead of bytecode + disassembly (just have to change this part at the top) and then stores the source (just have to change this part at the bottom). Then upon exiting the The robot interpreter will then see if bytecode exists for a robot as it is run, and if not it will assemble the source code into bytecode. There's no point storing both in the MZX file.

From here it'd be nice to see a new Robotic VM, or maybe if you're so inclined (and you get far enough with the interoperability) you can convert it to Lua, although I'm not sure yet if I'm happy with pushing Lua as the foundational language of MZX.

Nightwatch, on May 18 2008, 01:25 AM, said:

I'm thinking about moving to a modified version of the scroll/sign editor for Lua. It shouldn't theoretically be that hard to add a "Check Syntax" shortcut and some simple syntax highlighting for friendliness: Lua is very easy to parse. The only problem is that the scroll editor doesn't have copy/paste and other nice things... but one of the advantages of Lua is that you can read scripts from files on disk, so that isn't a hugely pressing matter right now, as far as I see it. (If you really want to use vi/Notepad/UltraEdit/whatever to edit your Lua, go for it!)

Thoughts?

Sorry, but I think this is a really bad idea. The scroll/sign editor/display code and the help system is probably the worst bit of code left in MZX, in fact, the former even has fatal bugs that haven't been killed (unless they have since I last worked on things). I meant to rewrite it entirely like I did most everything else, I just haven't bothered because really signs and scrolls aren't that important anymore and I don't really mind discouraging the use of the clunky [ boxes (I guess that's not really entirely fair of me though). These editors have just about zero functionality. If you're uneasy with the robotic editor then you should probably write a new one entirely.

You know, you can use external scripts for Robotic too.. :<

[ajs]

Haven't got masses of time today, so I've not been through the code very carefully yet, but I've started some minor janitorial improvements (necessary for merging) at this stage on your robo-ops branch. Please merge. The order of merging follows my description, NOT the alphabetical ordering of the patches.

Patches

robo_ops-tabs-to-spaces.diff
This one should be obvious -- your code was correctly indented if you had tabs set to 2 spaces, but I think you just forgot to transform these into spaces before pushing to your repository. There are NO CODE CHANGES in this diff.

idops-ht-remove-unnecessary-casts-from-void.diff
MZX isn't C++ any more so implicit casting to/from void* is possible (in C). Remove any extraneous casts bringing malloc/calloc usage in line with the rest of MZX's code.

exofy-style-remove-struct-namespaces-remove-trailing-whitespace.diff
Mostly your formatting is good, and I'm very happy you acknowledged this in your latest code release, but there's still some minor issues, mostly with keywords and Exo's dislike of whitespace. I've also removed trailing whitespace from a few lines. It isn't 100% ExoCode ™ but it's not far off. I also removed any explicitly named structs that weren't being used and pre-wrapped them in typedefs.

add-gpl-boilerplate-restore-exo-copyright-robo_ops.diff
Exo's copyright was never added to robo_ops, even though some of the code there has been copied verbatim from runrobo2.c. I've restored the copyright. I've also gone to the liberty of instating the GPL boilerplate and re-formatted your file descriptions. Consistency is king, right?

include-compat.h-wrap-with-decls.diff
All headers should include compat.h and be wrapped with decls. The decls are fairly pointless now, but historically they allowed us to mix C/C++ code, and forseeing that maybe happening again in the future I don't think it's a bad idea to preserve it. Including compat.h is essential for non-GCC (MSVC) compilation.

t-text-namespace-fix.diff
I noticed you'd already added a hack here for another data.h constant. This adds another which conflicts with a Windows header declaration "TEXT" on win32. I really think we either need to rename all of these en masse or go back to the enum[1]. This patch fixes mingw32 compilation.

msvc-stack-variable-fixes.diff
MSVC (Microsoft Visual C++) does not support the GNU/C99 extension taken from C++ allowing stack variables to be declared mid-scope. Since no other code relies on this, and I just got MZX compiling and running in MSVC, I'd rather variables were declared at the start of the new scope. This patch does that, but it needs a once-over from you to make sure I've not introduced bugs.

I'll try to get back to you on any other issues that crop up. So far, the hashtable implementation looks fine and readily reusable, and the runrobo2 de-bloat is quite pleasant. I think we'll have a better idea about the mergeability of this stuff once I've converted the remaining users of the old API over.

The resulting program has been tested as compiling/working on Linux, Win32 (via mingw32) and MSVC (on Windows).

--ajs.

[1] I'm not sure I understand your comment explaining why the enum was a bad idea. The type "char" is explictly always unsigned with MZX due to -funsigned-char on all platforms. Legacy code relies on this, so it's not an assumption we can easily remove. You're right that C enums aren't always 'int', they might be short or some other type, but that's what casts are for. Finally, an enum helps eliminate some of the namespacing issues.

[Exophase]

ajs, on May 18 2008, 07:46 AM, said:

[1] I'm not sure I understand your comment explaining why the enum was a bad idea. The type "char" is explictly always unsigned with MZX due to -funsigned-char on all platforms. Legacy code relies on this, so it's not an assumption we can easily remove. You're right that C enums aren't always 'int', they might be short or some other type, but that's what casts are for. Finally, an enum helps eliminate some of the namespacing issues.

C enums ARE always int, it's C++ that has the ambiguity (although there could be compiler options to override this)

[ajs]

I don't really want to argue semantics here, but technically that's wrong too. The C99 standard (6.4.4.3/2) specifies only that enumeration constants are 'int', but the size of the storage for an enum is "implementation defined". Googling for it I actually found a GCC bug which explains this ambiguity.

Of course, the implementation definition will almost always be "int" too, but if you look at 6.7.2.2/4 in the standard it explicitly says:

Quote

Each enumerated type shall be compatible with char, a signed integer type, or an
unsigned integer type. The choice of type is implementation-defined,110) but shall be
capable of representing the values of all the members of the enumeration. The
enumerated type is incomplete until after the } that terminates the list of enumerator
declarations.

It's ambiguous, but I take this to mean that if we provide -funsigned-char, we are changing the implementation definition of char to be "unsigned char", and therefore the enum must be compatible with it. So literally char c = enum_t; should work without truncation (where enum constant < 0xff). This of course assumes that GCC isn't promoting the type to int unconditionally, as you asserted. I have to agree that I think it probably is doing just that.

I guess Nightwatch just needs to explain more about what exactly he found the issue to be.

--ajs.

[Nightwatch]

ajs, on May 18 2008, 01:08 PM, said:

It's ambiguous, but I take this to mean that if we provide -funsigned-char, we are changing the implementation definition of char to be "unsigned char", and therefore the enum must be compatible with it. So literally char c = enum_t; should work without truncation (where enum constant < 0xff). This of course assumes that GCC isn't promoting the type to int unconditionally, as you asserted. I have to agree that I think it probably is doing just that.

I guess Nightwatch just needs to explain more about what exactly he found the issue to be.

--ajs.

GCC makes the mzx_thing enum an int unless you use __attribute__((packed)) or -fshort-enums. Unfortunately, MSVC has neither of those, and a quick google turned up some people moaning about how MSVC doesn't have these. So either I had to change it into a #define or throw out the assumption that level_id is a collection of mzx_things. I was getting crashes relating to exactly that problem. I really like the idea of the board being a set of mzx_things, personally, because it seems wrong that the type mzx_thing is 4 bytes, while the actual things on the board are 1.

Another problem is that MSVC doesn't have stdint.h, so you can't use uint8_t and so forth. I learned this only late, and I've been trying to convert them to Uint8, etc, but I'm certain I've missed a few.

Quote

Sorry, but I think this is a really bad idea. The scroll/sign editor/display code and the help system is probably the worst bit of code left in MZX, in fact, the former even has fatal bugs that haven't been killed (unless they have since I last worked on things). I meant to rewrite it entirely like I did most everything else, I just haven't bothered because really signs and scrolls aren't that important anymore and I don't really mind discouraging the use of the clunky [ boxes (I guess that's not really entirely fair of me though). These editors have just about zero functionality. If you're uneasy with the robotic editor then you should probably write a new one entirely.

You know, you can use external scripts for Robotic too.. :<

I'm going to try writing a new one entirely, depending on how quickly I can whip something up. It won't be too feature-rich at first, but that's to be expected. I'll try to make it language agnostic, so that eventually it can be used for Robotic as well, in the way you suggested.

[ajs]

My MSVC patch converts all the uint8_t's over; if we find other places where these types are needed we can always engineer backwards compatibility for it. As for MSVC's (and GCC's) behaviour WRT enums, I'm still not sure I fully understand the issue. Sure, the board is a bunch of chars, and sure, the enum might be an int, but at the point where the board is accessed, via a macro or a function, surely we can just cast the enum? The enum's still good from a maintainability point of view because it's a) not preprocessor magic, so the names don't conflict as much and b) it's kinda type-safe.

So could you explain why exactly you have to use the enum LITERALLY for representing the board/layer data?

EDIT: Okay so in detail I'm having difficulty understanding why we can't have something like this (all pretty much pseudo code);

char internal_level_thing[x][y];

if (internal_level_thing[0][0] == MY_ENUM_CONST) // this works fine

internal_level_thing[1][1] = MY_ENUM_CONST; // this should work too, if you get warnings, stick a (char) before it

Or use the enum thing I posted (I don't see any reason why the assembler should be horrid). Or write an accessor function inline:

static inline mzx_thing get_board_at(int x, int y)
{
  return (mzx_thing)internal_level_thing[x][y]; // this should zero extend from char to int in the register
}

static inline void set_board_at(int x, int y, mzx_thing t)
{
  internal_level_thing[x][y] = (char)t;
}

Etc etc.

I guess I must be missing something obvious or this looks like a non-issue to me.

--ajs.

[ajs]

Dunno how portable this kind of crap is, but it might make some of the logic simpler:

#include <stdio.h>

union {
		enum {
				BLAH_1 = 0,
				BLAH_2 = 127,
				BLAH_3 = 255,
		} e;
		char c;
} both;

int main(void)
{
		both.e = BLAH_3;
		fprintf(stderr, "%d %d\n", both.e, both.c);
		return 0;
}

Compile it with -funsigned-char. GCC does the right thing.

--ajs.

[Nightwatch]

It isn't really an issue in the way you suggested: I mean, you can make it work. You can make anything work: it's C.

It's more of an abstraction issue: it would be nice if level_id really was an mzx_thing to provide a clear hint that level_id describes things. I just don't like the way you have to cast (i.e. defeat the type system) just to avoid wasting memory in this instance.

[ajs]

EDIT: Actually, forget that. I don't buy the type safety aspect of enums anyway, because there basically is NO type safety in C for an enum. GCC won't moan even at the highest warning level with any of the following examples:

enum A {
  BLAH,
};

static enum A test(enum A var)
{
  return var;
}

int main(void)
{
  enum A value = BLAH;

  int b = test(value); // no warning, even though retval doesn't match
  test(0); // no warning even though param doesn't match
  test(2); // no warning even though constant doesn't exist in enum

  return 0;
}

So it's a lot worse than C++ in this regard and that basically chucks the type safety out the window. Basically all it buys you is auto-generation of numeric constants (which MZX isn't using in data.h), disallows duplicate names or the same value with two different names (this is useful) and some self-documenting properties. Oh, and the namespacing issue, which is pretty much the only thing I care about.

So there's two options -- either make the define names more ugly but less clashy, or go back to the enum and have some wrappers.

--ajs.

[ajs]

Okay, I've had a chance to look at this a bit more thoroughly. I can see which functions (now used only by the editor, not runrobo2) need to be switched to your new idops. I also fixed numerous compile bugs when disabling the editor/audio. Some of the bugs were mine, and latent due to the way headers were being included.

svn-fix-header-bugs-causing-compile-failures.diff
This patch fixes bugs in MZX SVN's headers. It can cause compile failures if something includes audio.h without first including configure.h, which should not have been possible. It also cleans up some dependencies in macro.h/configure.h that I spotted. I've merged this with trunk, but you can apply it separately if you like.

robo-ops-compile-fixes-with-disabled-editor-and-cleanups.diff

• Fixes compilation of your new robo-ops stuff when CONFIG_EDITOR is not defined;
  
• Moves some functions now used only by the editor to edit.c (from runrobo2.c). These will be targeted for replacement shortly.
  
• Some minor whitespace changes

Please apply to the robo-ops branch.

BTW I'm definitely looking to merge and test robo-ops before I release 2.82, so Lua experimentation after that point should gradually become easier. I might not add the Lua stuff there, but don't be discouraged by that because if things keep going at the rate they are going, it's very possible it'd be in some other version soon thereafter; also, feel free to keep doing releases yourself. TODO:

• Resolve enum vs defines problem and fix it in robo-ops;
  
• Refactor editor to use new idops and delete the legacy code;
  
• Test it

--ajs.

[Xx*SaturnineMatthias*xX]

I indirectly have this to thank for my being here right this second.

Oh well.

Suppose I need to learn Lua now. >.>

[asiekierka]

...

the hell froze over.

[ajs]

Terryn sent me a few backtraces from the robo-ops branch (failure to load worlds on win32). I found a few problems in the hashtable implementation:

fix-hashtable-impl-bugs.diff

• Fixes bug in hash_table_iter_next() which was potentially accessing memory it didn't own. iter->entry cannot be set to a value beyond iter->table->buckets[nbuckets - 1], but in some cases was being set to iter->table->buckets[nbuckets], and then being checked for NULL (which it might not necessarily have been). Though an easier fix was possible, I think this fix is clearer;
  
• The key should not be const void * as it refers to non-constant data. In this case I need key to simply be void* to free it later;
  
• calloc() is calloc(num_members, size) not calloc(size, num_members); minor but I noticed and switched it;
  
• Plugged some memory leaks. destroy_hash_table() now frees tab->buckets and also frees the key and value provided by the user. From a purist perspective, this might not be the best place to put the frees, but it suffices for the moment, and I've stuck a comment in justifying further enhancement later.

Please either apply, or ask me to rework this. Fixes robo-ops on win32.

--ajs.

[Nightwatch]

ajs, on May 19 2008, 05:01 PM, said:

Terryn sent me a few backtraces from the robo-ops branch (failure to load worlds on win32). I found a few problems in the hashtable implementation:

fix-hashtable-impl-bugs.diff

• Fixes bug in hash_table_iter_next() which was potentially accessing memory it didn't own. iter->entry cannot be set to a value beyond iter->table->buckets[nbuckets - 1], but in some cases was being set to iter->table->buckets[nbuckets], and then being checked for NULL (which it might not necessarily have been). Though an easier fix was possible, I think this fix is clearer;
  
• The key should not be const void * as it refers to non-constant data. In this case I need key to simply be void* to free it later;
  
• calloc() is calloc(num_members, size) not calloc(size, num_members); minor but I noticed and switched it;
  
• Plugged some memory leaks. destroy_hash_table() now frees tab->buckets and also frees the key and value provided by the user. From a purist perspective, this might not be the best place to put the frees, but it suffices for the moment, and I've stuck a comment in justifying further enhancement later.

Please either apply, or ask me to rework this. Fixes robo-ops on win32.

--ajs.

Whoops, those were my bad. The hash table implementation was very quick and dirty. Thanks for fixing it.

Now for a big issue. I talked to Exo about Robotic and Lua, and his opinion was that to do Lua right, it needs to replace Robotic. In other words, it should obsolete the Robotic interpreter entirely instead of providing an alternate interpreter. After giving it some thought, I realized that there's a fairly straightforward mapping between Robotic and Lua. The key is tail calls: Lua is guaranteed to perform tail call optimization (I have tested using a quick mock-up and verifying the disassembly), so any Robotic program can be broken up into a series of basic blocks (in the compiler theory sense) that tail call one another. Robotic makes this easy, because all labels are explicitly named (except for the top of the program). The tricky thing is ZAP and RESTORE, but these can be emulated using an array of functions and an index into that array.

So here's the big question: Should I work on replacing runrobo2.c with Lua? In this scenario, Robotic would still be fully supported, but it'd be translated to Lua internally instead of being executed directly. This would be a big change, so I welcome discussion.

[zzo38]

Nightwatch, on May 19 2008, 09:36 AM, said:

Whoops, those were my bad. The hash table implementation was very quick and dirty. Thanks for fixing it.

Now for a big issue. I talked to Exo about Robotic and Lua, and his opinion was that to do Lua right, it needs to replace Robotic. In other words, it should obsolete the Robotic interpreter entirely instead of providing an alternate interpreter. After giving it some thought, I realized that there's a fairly straightforward mapping between Robotic and Lua. The key is tail calls: Lua is guaranteed to perform tail call optimization (I have tested using a quick mock-up and verifying the disassembly), so any Robotic program can be broken up into a series of basic blocks (in the compiler theory sense) that tail call one another. Robotic makes this easy, because all labels are explicitly named (except for the top of the program). The tricky thing is ZAP and RESTORE, but these can be emulated using an array of functions and an index into that array.

So here's the big question: Should I work on replacing runrobo2.c with Lua? In this scenario, Robotic would still be fully supported, but it'd be translated to Lua internally instead of being executed directly. This would be a big change, so I welcome discussion.

No, I don't agree with that. I don't know whether it would be slow (probably not slow if it is pre-compiled to Lua bytecode), or any other bugs, but with enough time you should be able to fix it anyways. Still, I disagree with that. Just keep it separate, and create a Robotic editor and a Lua editor. However, it is good that you can do it (with tail calls), so possibly also add a converter, to convert from Robotic to Lua. You could have a configuration option to do it automatically or manually. In my opinion, regardless of what you are using (Robotic, Lua, Forth, built-ins, etc) you should still be able to use Robotic as well (so that you can still do it the old way if you prefer, or if you have to, or for whatever other reason), and combine Robotic and Lua in the same world as well, in case you want to use both.

[Frobozz]

zzo38, on May 19 2008, 03:57 PM, said:

No, I don't agree with that. I don't know whether it would be slow (probably not slow if it is pre-compiled to Lua bytecode), or any other bugs, but with enough time you should be able to fix it anyways. Still, I disagree with that. Just keep it separate, and create a Robotic editor and a Lua editor.

Having two separate editors would just lead to confusion and waste developer/maintainer effort.

zzo38, on May 19 2008, 03:57 PM, said:

However, it is good that you can do it (with tail calls), so possibly also add a converter, to convert from Robotic to Lua. You could have a configuration option to do it automatically or manually. In my opinion, regardless of what you are using (Robotic, Lua, Forth, built-ins, etc) you should still be able to use Robotic as well (so that you can still do it the old way if you prefer, or if you have to, or for whatever other reason), and combine Robotic and Lua in the same world as well, in case you want to use both.

The idea behind the converter isn't to do away with Robotic. Backwards compatibility demands that Robotic still be available. Rather it would allow for the Lua VM to replace the Robotic VM. Amongst other benefits, it would allow for Robotic to be easily expanded with more capabilities (such as true functions as opposed to the current subroutine hack that exists).

[commodorejohn]

Well, if zzo38 opposes it, that's reason enough to go ahead with it.

[zzo38]

commodorejohn, on May 19 2008, 01:29 PM, said:

Well, if zzo38 opposes it, that's reason enough to go ahead with it.

No it isn't, you need to vote. Make a poll for voting it and then you can go ahead with it.

[mzxgiant]

zzo38, on May 19 2008, 04:32 PM, said:

No it isn't, you need to vote. Make a poll for voting it and then you can go ahead with it.

I'm not sure why you're so opposed to this. As far as I can tell, this doesn't involve a huge front-end change. It seems to me like the majority of the changes being proposed are all under-the-hood and are there for the purpose of future development of functionality.

[Xx*SaturnineMatthias*xX]

Although I have absolutely no idea how to program in lua, I'm fine with this so long as the robotic frontend stands.

[zzo38]

mzxgiant, on May 19 2008, 12:37 PM, said:

I'm not sure why you're so opposed to this. As far as I can tell, this doesn't involve a huge front-end change. It seems to me like the majority of the changes being proposed are all under-the-hood and are there for the purpose of future development of functionality.

If you are the developer then you can vote more than other people. (Or of course you can just approve of it immediately and add it in, since you are the developer, but I don't know if you necessarily should.)

[Xx*SaturnineMatthias*xX]

If enough people don't want it, eventually someone will fork.

[asgromo]

everyone shut up except for ajs, Exophase, and Nightwatch

[Exophase]

Nightwatch, on May 19 2008, 12:36 PM, said:

So here's the big question: Should I work on replacing runrobo2.c with Lua? In this scenario, Robotic would still be fully supported, but it'd be translated to Lua internally instead of being executed directly. This would be a big change, so I welcome discussion.

You know my vote.

Glad to see you got the thing with tail calls worked out. You'll need a hash of functions (I'll assume here you mean references to functions though, but that's usually what a closure ends up being, not like you can't assign that to a variable) to support indirect gotos as well, ie goto "this&blah&" - fortunately you can pretty easily see what existing labels can actually match that instead of throwing them all on there. The only other really weird thing about Robotic I can think of right now is [ box support, since you can do stupid things like this:

[ "split box"
: "touch"
[ "gets split"
end

That'll display "split box gets split" at first, then just "gets split" when you touch it. [ boxes are annoying anyway, but usually they do end up being one command split across several. For this case you can probably just merge them and.. I don't know, some kind of magic. Maybe best to handle this at the Robotic level. I don't know if very much actually does this.

[ajs]

I'm with Exo, I think it's a good idea. Regarding the issues with the editor, I've already mentioned to Exo that I think writing a wholly new editor at some point using a modern toolkit would also be a good idea in the long run, but if you're prepared to hack on the existing one then I don't think there's a problem.

I would say that we NEED to be incremental about this. If it's not done carefully and with good attention to backwards compatibility, IMO it's not worth doing. I think a lot of the editor changes could happen without Lua (such as moving the editor from "one bytecode line per string" for robotic, compile on exit, etc.), and so obviously I'd recommend this approach.

Nobody has done much with MZX in recent years and it needs an invigorating change, so I'm all for you doing this if you have the time.

Also, I'd appreciate the people who sensationalise everything in this (and other) threads not doing so. We don't need idiots every 5 minutes telling us about their marvelous forks. The reality goes without saying -- the community will use whatever they want to use. Nobody with any real ability is going to waste their time working against that.

--ajs.

[Nightwatch]

Exophase, on May 19 2008, 10:18 PM, said:

Glad to see you got the thing with tail calls worked out. You'll need a hash of functions (I'll assume here you mean references to functions though, but that's usually what a closure ends up being, not like you can't assign that to a variable) to support indirect gotos as well, ie goto "this&blah&" - fortunately you can pretty easily see what existing labels can actually match that instead of throwing them all on there.

Lua namespaces are hashes of functions. mzx.shootseeker() is the same as mzx['shoot' .. 'seeker'](). (.. is the concatenation operator.)

By the way, I was wrong in IRC about Lua global optimization. Lua precomputes all hashes for constant strings, including the names of global variables referenced in the program, so basic global variable accesses are actually optimized to O(1) modulo chaining.

Quote

The only other really weird thing about Robotic I can think of right now is [ box support, since you can do stupid things like this:

[ "split box"
: "touch"
[ "gets split"
end

That'll display "split box gets split" at first, then just "gets split" when you touch it. [ boxes are annoying anyway, but usually they do end up being one command split across several. For this case you can probably just merge them and.. I don't know, some kind of magic. Maybe best to handle this at the Robotic level. I don't know if very much actually does this.

I'll just have a "box buffer" that [ and friends will add to, and then there will be a "box()" command that gets statically generated at a box-command-to-non-box-command boundary in order to display and flush the buffer. Labels won't generate such a boundary. Something like this needs to be done anyway to get the counter-sensitive box commands to work, so it's not that weird.

[Nightwatch]

I'm in the middle of writing the expression parser. Expressions will be compiled to static single assignment. Since Lua is a register-based VM, this should be efficient. String-number conversion will be done lazily.

For the recompiler, I needed a bunch of string functions, so I went ahead and created a simple String object (pointer+length model) that can be found in string.c. It's simplified the code dramatically.

While writing the recompiler, I unearthed some critical bugs in my hash table iteration function. I cherry picked the fix from the recompiler branch into hash-tables and lua. Please pull when you have the chance. Also, hash tables can now specify custom destroy functions, as you suggested (and as I needed for String objects).

[ajs]

I merged it back into my SVN branch here, but your robo-ops branch doesn't actually compile ATM. You might want to check that out so I can be sure I merged correctly.

--ajs.

[Nightwatch]

It's certainly quite possible that robo-ops doesn't compile: I was doing lots of merging between branches and I didn't test them all. I'll get on that when I get some time.

Further development on the recompiler needs to wait until all Robotic functionality is available in Lua. And that in turn needs to wait for the editor.

I've been sitting on some code I wrote for a new, language-agnostic text editor. It already supports joining lines, splitting lines, and more natural scrolling (not always in the center). Exo doesn't think that these are good enough reasons to justify rewriting his Robotic editor, and I agree. But the new String objects offer a really clean way to do language-agnostic editing: just associate with the editor a language backend that annotates a String object with syntax highlighting in the usual MZX way (~ and @ color codes). Since String objects resize dynamically, this should be easy to do.

Unfortunately, retrofitting the existing Robotic editor to use String objects would basically involve rewriting it. So I think there may be a case to use a new one. I encourage discussion on this though.

[Frobozz]

How about some screenshots of the new editor? Or will it look the same?