U7 Network Programming

[wintermute_turing_ai]

Hi folks. I've spent some time searching the forums for all of the 'u7 online' threads - so excuse this post if I've glossed over anything.

Recently my girlfriend started playing U7, and I'd really like to adventure with her as she explores the game. While a full-featured multiplayer system (a la Diablo) would be great, I'd be more than happy just to have our characters walk around in the world simultaneously. As such, this is a technical thread - and not a gameplay thread.

I'm interested in writing some simple network code (likely using SDL_net or one of the easier abstraction layers such as "Net2") in the classic host/client configuration of games like Diablo.

Keep in mind that I have not perused the exult source yet, however I was hoping someone might help focus my efforts:

- are there any specific technical reasons that network code cannot be integrated into the exult source tree?
- how are events handled? (ie, is there an event handler class?)
- what might be the best 'insertion point' for some network code? (ie, an extension of the event class?)

Basically I'm just trying to understand how the Exult source tree is organized, and which source files I should be looking at first in order to add a network class that transmits Avatar x/y/z coords between 2 machines running Exult.

thanks much for your time!
WM.

[wintermute_turing_ai]

Nobody with insights on the source tree? I'd really love to get going on this project. Since I'll be working on this alone, obviously I won't be making this the most complex patch - just some simple network code.

cheers
WM.

[marzo]

I think that nobody answered because of two reasons:
(a) They aren't very interested on multiplayer Exult;
(b) They know that adding multiplayer to Exult would be like trying to fit a square cylinder into an hexagonal pyramid-shaped hole with nothing but a screwdriver. Or something like that.

More seriously: Exult is simply not structured to a multiplayer environment; not with a "quick" hack like you want, anyway: it would take a comprehensive restructuring of the code to pull it off. Off the top of my head, I can think of at least a dozen places where you would have to add multiplayer code of one type of another -- party management functions (several places, as there are at least 3 versions of the code I found which get used with different configurations), when attacking objects/actors, when projectiles reach their targets, when receiving input (there are several different input handlers used in different circumstances -- e.g., when you are in a conversation vs when you are walking around vs when you are in the cheat menu), when defining the camera actor, when NPCs/objects move/are moved... Not to mention that you would have to find a way to network the usecode machine (or have it centralized in a "server"), conversations, gumps (or enforce the "gumps don't pause game" option in multiplayer mode), etc.

I am not trying to put you off; if you are determined, go for it. I am just warning that it would take a huge amount of work.

[Skutarth]

(a) Nonsense. Plenty of people want multiplayer in Ultima 7. Just because they didn't post here doesn't mean they don't exist.
(b) Nothing is impossible. In fact, restructuring the code to be more object oriented would help make a project like this easier, and it would most likely be accepted into the main trunk with open arms.
Maybe if I have time around Christmas break I could help out. Right now, though, I'm juggling college and several coding projects already.
Until then, good luck.

[drcode]

As others have pointed out, this will not be easy. But there is 'network' code already in Exult for communicating with ExultStudio, and it's in the 'server' directory. Note that it's very Unix-oriented and assumes that the client (ES) and Exult are on the same machine, but only because that's what I knew at the time.

[marzo]

Nonsense. Plenty of people want multiplayer in Ultima 7.

Not what I said; what I said was that some of the people with the knowledge to answer his question didn't because they didn't want multiplayer U7. I myself fit into that category.

Nothing is impossible.

I never said it would be impossible, just that it would take a lot of work; you missed the more than 2/3 of message where I outline why it would be difficult but doable.

In fact, restructuring the code to be more object oriented would help make a project like this easier, and it would most likely be accepted into the main trunk with open arms.

If someone has the time and dedication to do this, I would be at the head of the welcoming committee. There is, however, a lot that could be done even in the context of non-multiplayer Exult -- I, for one, have been considering an over-hauling, recentralization and consolidation of the file handling system regarding patch files; a consolidation of path-finding for actors or all sizes and types; a consolidation of party formation and management; and a few others. These things take time, though, which is the real problem.

[torgus]

why add them into the party. have them appear as another NPC. however it would still takes tons of work. good luck i'd play it multiplayer!

[marzo]

why add them into the party.

For things like stealing and such: NPCs only attack party members... Also, so that NPC party members (if any) do not suddenly start attacking the other players because they (the NPCs) run into the area of that explosion spell they (the other players) cast.

[wintermute_turing_ai]

Ah - thanks for the replies so far folks. I will start hunting through the source files and tracing out execution - even I just manage to transmit the *tiniest* bit of data between a client-server setup (ie. party position, updated via the existing handlers), I would be satisfied. As I said before, I'm not looking to restructure the entire codebase, and if it does in fact entail that - then I'll be happy to call it quits because I'm a developer myself with plenty of other projects to keep me occupied

As for submitting it to the trunk, I doubt that is something I would ever do, since my coding style is not likely amenable to the existing codebase. This is just an experiment.

thanks again for the replies!

[drcode]

Have fun with it. That's the beauty of open-source.

[Skutarth]

Marzo: I'm sorry that you don't know that "nothing is impossible" is an idiomatic phrase, but you don't have to respond so condescendingly.

And you should bother reading the rest of (a): "Just because they didn't post here doesn't mean they don't exist."

[marzo]

Marzo: I'm sorry that you don't know that "nothing is impossible" is an idiomatic phrase, but you don't have to respond so condescendingly.

Sorry about that.

And you should bother reading the rest of (a): "Just because they didn't post here doesn't mean they don't exist."

I did; it doesn't change in the slightest what I wrote in either post. To clarify, here is the reasoning behind my first post: I know that:
(1) there are people that want multiplayer U7 (I saw a few of the many threads where people asked for it in the past, and I know there is an entry in the FAQ);
(2) there are people with the knowledge of the Exult codebase to answer wintermute's query;
(3) there are people simultaneously in set (1) and in set (2);
(4) there are people in set (1) but not in set (2) and vice versa;
(5) Exult is not structured for multiplayer and it would be a major task to implement multiplayer in Exult;
(6) that the people in set (2) would agree with me about (5), whether or not they are in set (1).

The logical conclusions of those points are the ones I listed in my first post; namely, that people did not reply because they did weren't interested in multiplayer (i.e., were not in set (1)) or that they were interested in multiplayer Exult, had the necessary knowledge of the Exult code base to answer wintermute's query, agreed that it would be difficult and decided not to post (likely) because it would be too much work to give a thorough answer. I will grant that I missed the cases where (c) people wanted to post a reply but left for a later time and forgot about it and (d) people wanted to see what others would post before they posted themselves.

[Skutarth]

Whatever. Stop wasting your time beating a dead horse and spend more time helping to make Exult more awesome like you have already been doing.

[Dominus]

now who's condescending.

[Skutarth]

Right... I can't see how an obvious joke followed by a compliment can be considered condescending.

[Dominus]

Skutarth, if you have nothing more to add on topic, please stay out of the thread as will I or I'll just clean it.

[Andrea B Previtera]

Wintermute: One (silly but fun) thing that would be easy to code into the source tree, given your initial proposition - would be a "guardian mode". Imagine that someone (ie. your girlfriend) is playing Exult on machine "A". Machine "A" also acts as a server, which can be polled for current game stats, party position etc. - The client machine "B" (ie. yourself) can connect and reproduce on your screen what's going on on machine "A", making you a spectator with some powers. For example you may code in the ability to cast realtime eggs on the client - making monsters appear, music play, or something else. Silly indeed, but it's a start - and by a first look I think it would take to insert network code only in 2-3 places.

[Madoc]

There is an Oblivion "multi-player" mod that is still in development. Although the Oblivion game itself does not support multi-player, it uses a very simple trick: The other player(s) are NPCs, which are "remote controlled" by the multi-player server. So, every client is in fact single player. It's just that all the other players are NPCs on every client, and their movements etc. are synchronized through the mod by the server.

When random events in Exult can be controlled by, say, synchronizing the random number seed throughout all clients, then even the world's natural development should be the same on all clients. (Not that easy, I know...)

And when the server can "remote control" not only NPCs, but all objects of the Exult game world, then it should be quite possible to have a consistent world state. And with this approach, it really does not matter at all if the game is "fit" for multi-player or not. Of course, it has drawbacks: Every client could hack the game and cheat by watching the other players when they should not be able to, for example. But for this limited purpose, we can just assume that the players are honest.

I think stealing/"other players as party members" should be the least problem. First, one would need a working Exult-plugin for the "remote controlling" feature. The next step would be a rudimentary server that synchronizes the client's states.

Not a small piece of work, but possible.

[marzo]

There is an Oblivion "multi-player" mod that is still in development. Although the Oblivion game itself does not support multi-player, it uses a very simple trick: The other player(s) are NPCs, which are "remote controlled" by the multi-player server.

While I never played Oblivion and didn't know about this multiplayer mod, this trick is what I was thinking about when I said it wouldn't be easy to add multiplayer to Exult; implementing this trick would require either, an overhaul of the source tree with multiplayer in mind or network code in dozens of places.

[drcode]

Andrea: That's a pretty clever idea. The network code used by ES would have to change so that it uses a port instead of a file socket, and we'd have to allow ES to work while the game is still in play. Plus, Exult would have to be running on both machines, on 'A' for playing and on 'B' for editing. I guess this would be kind of tricky.

[Andrea B Previtera]

Yes it's probably the most feasible thing (and still it's quite tricky) but doesn't really add much. It would be funny for... say... half an hour As for the rest of the question, let's stop lying to ourselves: everyone would love to network play U7. I can imagine the official Exult shard constantly filled by all the memebers of the forum and it's really a nice view. "Where are you DrCode?" "Oh I'm in Vesper double clicking a beer with Marzo, join us". The problem, is that such a beautiful dream can't be built on hacks: it requires a total rewrite of the engine, with a lot more multithreading - and the obvious network code. .... Exult 2.0?

[marzo]

and we'd have to allow ES to work while the game is still in play

Or a "hack" could be made that allows the game to be in map-edit mode in a client but not in another (with the server seen as another client for these purposes).

[marzo]

it requires a total rewrite of the engine, with a lot more multithreading - and the obvious network code. .... Exult 2.0?

Make that 3.0 or more; there is enough stuff yet to be done enough to make Exult 2.0 and more before even thinking or the multiplayer code.

[Malignant Manor]

Wouldn't it just be easier to do a mod for UO instead? It is already made to have as many people as a server can handle and has the ability to have multiple people be in control of game master only functions. The only thing that Exult being online would accomplish most of the time is being an online chat. Ultima 7 definitely isn't balanced for multiplayer anyway.

[SB-X]

That's not really the point...
(and outside the scope of the original question)

I think he got his answer already though.

[Colourless]

I'll give my input here. Exults method of doing event timing makes it very hard to do syncronization in the game itself. Just look at how may times syncronized animated shapes have been broken and then fixed. Exult is also very susceptable to differences in machine speed. It doesn't attempt to keep rigid timing like games, and in particular networked games, usually do. This will make it very very hard to do cross network syncronization. It's also what is the major issue preventing things like smooth scrolling. Smooth scrolling would be cool as hell to have in the engine, but Exult just isn't coded the right way to make it easy.

-Colourless Dragon

[drcode]

What should we do differently then? Are we limited by the 'palette rotation', which modern games don't have? I wonder what it would take, assuming it's possible, for Exult to work like a modern 3D game (without the actual 3D, of course).

It's been a long time since I've worked on the core engine (or much of anything...), but my guess is that it is the palette-rotation that limits the engine. Perhaps we could get rid of that (since nobody runs in 8-bit color anyway), and treat shapes with palette-rotation colors like we do the animated objects.

[marzo]

I wonder what it would take, assuming it's possible, for Exult to work like a modern 3D game

Does anyone know if Exult 3D does anything differently? We could steal a few ideas if it does.

but my guess is that it is the palette-rotation that limits the engine.

From everything I've seen, the palette rotation is but a small part; and it could modified to be done asynchronously, so it is not that really a big deal. The real issue is that everything in Exult (except the audio subsystem) is synchronized to the game clock's "ticks" -- even the framerate; this should not be needed to synchronize animations at all*. And it is, in part, what prevents smooth scrolling -- hard to be too smooth at 10fps.

* = And, in fact, it prevents some things which "should" be possible -- for example, the faster missiles can be drawn only once or twice in their path, if at all, due to this limitation. It *looks* bad...

[Andrea B Previtera]

Since the topic drifted to "what limits exult" - I think the engine is also bent on itself for what concerns file formats. Of course it started as a replacemente for "u7.exe" - and that meant reading the original formats and using them as they were. But now, nothing should avoid simply converting all the graphics to plain true color png files and recode the engine to use those instead of doing all that unpacking, seeking, and reading exotic proprietary formats (that also seem to stop a lot of people from creating new stuff). Most of other formats, too, could be opened to plain text files: items, eggs, npcs. The first step in this direction, after all, was creating the sound packs and ogg soundtrack.

[marzo]

But now, nothing should avoid simply converting all the graphics to plain true color png files

Now, *that* is something which is limited by palette rotation .

Most of other formats, too, could be opened to plain text files: items, eggs, npcs.

That is something which I already intend to do down the road, yes.

[drcode]

"The real issue is that everything in Exult (except the audio subsystem) is synchronized to the game clock's "ticks"..."

I think the reason for that is to keep multiple animations like the shoreline tiles in sync. But it looks like the frame-animators synchronize themselves by rounding the delay to 1/100 sec.

I'm playing around today by commenting out the palette-rotation in exult.cc, and also the line "ticks > last_repaint + 50 ||", so that we only repaint when the "painted" flag is set. And I'm also setting that flag in "gwin->add_dirty()". The shoreline animations still look good this way, but I also want to add some code to show the framerate (probably the only thing I'll check in).

[drcode]

Oops, that wasn't right. I don't want to set the 'painted' flag. Instead, I just want Exult to paint when 'dirty'. There's a comment on the original condition about avoiding 'jumping walking', but I don't notice any jerkiness without it.

[wintermute_turing_ai]

Andrea: Thank you for the suggestion. That indeed might be a great place to start, if not simply for a "proof of concept". I'll look into it this weekend.