Metaverse Ink

As OpenSim becomes more stable, many people are looking into developing new businesses on top of it. That is great! An ecosystem of businesses is exactly what we want to happen, if OpenSim is to fulfill its potential. However, let me add a word of caution in the form of a personal plea regarding doing businesses on top of OpenSim: be careful with what you patent, or you may put yourself out of your business before you even create it by placing obstacles on the way of the infrastructure.

Ever wondered why it’s taking so long for a web of virtual worlds / 3D web to emerge? Here’s a fact: there’s plenty of wonderful game engines and MMO platforms that have one thing in common: they are all proprietary. Not only that, but they aim at being the only game in town, all 20 of them. Some of them are very successful in their own businesses, but none of them will become the common infrastructure for interoperable virtual worlds. Infrastructures must be owned by everyone and no one in particular.

Enter OpenSim. At this point, OpenSim is one of only two open source virtual world platforms with which one could imagine developing that much needed infrastructure for immersive interconnected virtual worlds to bloom. (The other one is OpenCobalt, a Smalltalk platform; Sun’s Wonderland has just been discontinued by Oracle, and its future is unclear) OpenSim has been created from day 1 with the goal of becoming the Apache server for virtual worlds. It’s still young, so there’s still a few things to do in order to accomplish that goal. Please let us finish! Don’t start creating obstacles by filing patents that are infrastructural in nature. Those patents will be a nuisance, and ultimately useless for your business because one of two things will happen: either (1) what you patent is so important that it will be critically missing from the common infrastructure because of your patent, therefore the infrastructure will never happen; or (2) what you patent can be done in a different way, in which case that other way will make it to the common infrastructure, and your patent-based business will miss the point.

I know that investors, before they invest, and managers in large companies, want to know where your own beef is when you’re dealing with an open source platform. In the US, “beef” usually means protected Intellectual Property in the form of patents.  Some large companies are particularly aggressive when it comes to filing patents on absolutely everything for which a patent hasn’t been filed yet, no matter where the idea came from, and no matter the merits of the claims; in those companies, employee compensation is, in large part,  based on the number of patents filed, so the employees quickly learn how to play the patent game (this practice is an aberration, and one of the reasons why I decided to take a job in Academia). I don’t have a simple solution for this need to patent, but just this general advice: patent OpenSim-related infrastructure at your own peril. For starters, you won’t gain any friends among some of us core developers (me, at least). You’re putting the whole effort in jeopardy. I will use every resource I have available in the UC in order to shoot your merit-less patent applications down. Second, I can guarantee you that if something you patent is with merit but critical, I won’t stop until I find another way of doing it that doesn’t step over your patent.

What else can you do to make your investors/managers happy? Here are some options:

a) Code faster. Be the first to have the implementation of something important, then you can try to grab the market before the open source solution comes around. But no patents, please — see above.

b) Create interesting, engaging services. If people like what you provide, they will come.

c)  If you really must, patent methods for doing something faster/cheaper/better. Optimizations of published processes are always desired by potential customers.

d)  Again if you really must, patent specific applications of the infrastructure that focus on the vertical that you are trying to make a business on.

Needless to say that this post is my own view of things and in no way reflects any official position of the OpenSim core developers.

The OpenSim Library has been a neglected part of OpenSim since I can remember. Apart from a few animations and example scripts, there hasn’t been anything interesting there. Even though it was possible to add things to it, the process was so cumbersome and error-prone that no one cared to do it. So the Library stagnated. That stagnation has finally come to an end.

As of today, it is now possible to add any arbitrary elements to the OpenSim  Library simply by using IARs. I’m going to explain it below. For starters, the latest release of the Diva Distro already has some new freebie content in the Library (see picture above — zoom in). I added three outfits, 2 female and 1 male, that you can use to replace the horrible Ruth avatar; and I added a very small collection of objects. They are available to everyone with the latest diva distro, diva-r11766. (Remember that to use the content in the OpenSim Library you first must copy it to your own inventory, and then wear it or rez it.)

I have a larger object collection available from the github site. I didn’t include this in the distribution, because it’s quite large, larger than the distribution file itself. You can download it and place it under bin/Library if you want.

You can add more IARs to your Library, so that the content of your choice can be made available to all your users. The process is very simple: produce or get one or more IARs with your favorite content, and place them under bin/Library — that’s all. The content of those IARs will be added to the OpenSim Library automagically.

A few details. In general, IAR content is placed at the top level of the Library. If, however, if you are careful with the name of the IAR files, you may be able to control their placement better. For example, if you want to add things under, say Clothing Library, name your IAR something like “Clothing Library part1.iar”. When the first 2 words of your IAR file name match one of the existing subfolders of the Library, the content will be placed under the match. Content always adds up, so you can have several IARs, and they will all be loaded.

One important limitation: this only works for standalones, for the time being — which is what the diva distro is all about anyway. It will be nice to make this work for grids, but that will require substantial changes in the way IARs are coded.

Enjoy!

OpenSim now supports the concept of megaregions — several regions stitched together without borders. I love megaregions. For the kinds of applications that I am involved with, megaregions are a must — try sending traffic over a region border! (horrible). The diva distro comes with a megaregion.

However, megaregions have a few quirks and because of those quirks many people don’t use them. One of the major show-stoppers is importing content from existing non-mega regions. Yeah, it doesn’t work very well. But if you know the right incantation, it’s actually quite workable. I have been doing this regularly for the past 3 weeks, so I thought I’d make the incantation more public. I’m going to describe my exact situation — extrapolate for yours.

I am working on an application over a 3×3 region area. My team is building the content in a regular 3×3 region with borders, for historical reasons. The final deployment is in a 3×3 megaregion, since the application includes several dynamic behavior modules and I don’t want to deal with border crossings in them. So here is what I do.

Every few days I go to the console of my team’s building sim and save each region in an oar — that’s 9 oar files. I bring them to my development machine. Then I do the following steps:

1. Clear my DB from all existing prims. I do this directly on the DB with a SQL statement — DELETE FROM prims; DELETE FROM  primitems; Poof! there goes all content! If you’re starting from scratch you obviously don’t need to do this.
2. Start my empty 3×3 megaregion.
3. Import each oar file into the corresponding region: change region <region>; load oar <region.oar>. This is a bit tedious and makes me want to add something to OpenSim to handle this better. But, while that multi-region oar-importing support doesn’t exist, I have to do it one by one — 9 times in my case. Depending on how fast you type these commands, you may or may not be interrupted by OpenSim’s backup thread. This is a thread that writes new content in memory into the DB, it kicks in every minute or so. When this thread kicks in, you will see lots and lots of error messages, possibly in red, complaining about all sorts of things. Ignore it and type faster.
4. When I’m done loading my 9 oars I do the magic incantation on the opensim console: fix-phantoms
   That’s right, this is the magic incantation. Again, you will likely see lots of error messages flying through the console. Ignore them.
5. Then I wait for the next run of the backup thread, so that all prims are stored correctly on the DB. When the backup thread kicks in, you will see lots of debug (non-error) messages on the console saying that it’s writing objects in your SW region. Done!

Well, not quite. There’s another thing I do to clean things up. In megaregions, all prims are supposed to be in the SW region, and that region only, for example, in coordinates <345, 456, 40>. There should be no prims in any other region. However, I always end up with a few prims not being able to cross, for some reason that I still don’t understand. So I shut my megaregion down, I go again to the DB directly and I inspect the prims table again:
SELECT Name, GroupPositionX, GroupPositionY, RegionUUID FROM prims WHERE RegionUUID != ‘<SW region UUID>’;

On my regions I usually get a couple of hundred prims in this situation. Also on my regions, they all happen to have very suspicious negative coordinates. I know by experience that these prims will never be placed right, they’re just junk. So I delete them:
DELETE from prims WHERE RegionUUID != ‘<SW region UUID>’;

As it turns out, they don’t seem to make any difference, at least in my case.

Upon restarting the sim, I still see error messages on the console related to prims failing to cross. I haven’t been able to track what the problem is. But the good news is that when I login to it, all important content seems to be there in the right place.

This has been working very well for me for the past 3 weeks. It’s not perfect yet, but it’s quite workable.

Today, while we were doing load tests in Wright Plaza, I was also doing another kind of load test on my standalone. This one relates to server-side bots. I was able to have a 3×3 megaregion with ~2,000 prims, 200 bots and my client connected to it!

My client was quite happy. The only thing that didn’t seem to be working was the walking animation. Apart from that, I was able to walk, fly, chat and generally interact without much lag. Pushing it to 300 didn’t quite work well yet, I was stuck in 10, 10, 10 on login.

Mind you, server-side bots are a very light load on the server, much lighter than a regular client and lighter than a libomv bot. For starters, they don’t connect over the network, so there is no packet sending/receiving. Then, these particular bots I’m developing aren’t that smart yet, so they don’t request any assets from the server — their load is essentially the physics that comes with them and all the updates that are sent to regular clients as the bots move around.

I have been playing with server-side bots lately because I am involved in projects that require them for simulations. I will have to have at least 500 of these. We’re very close… I’m quite happy with the results today!

Goggle has announced its 3D Building maker for Google Earth. It looks really nice and simple. I think Google is getting that most people aren’t expert 3D modelers, and as such, simple tools that produce simple models will go a long way in modeling the entire planet.

Google’s intention is great at face value. However, I can’t help but wonder what will Google do with all that content produced by thousands of people around the world. If Goggle ever monetizes Google Earth, will it give back to the creators of those buildings? Or, like what Google does on the Web, will it take the content and run its own business without paying back to content producers?

We hit a milestone today in OpenSim. We piled up 52 avatars in OSGrid’s Wright Plaza sim, under 600M of RAM, and after an uptime of 10 hours. It eventually crashed, probably due to a lurking ultra-conservative lock somewhere in code. But hey! — this is fantastic news for OpenSim-based worlds. I can see 1.0 on the horizon.

Over the last few days, one of the key components of OpenSim, the code that deals with packets from/to the clients, has been replaced  with a new packet handler written by Intel’s John Hurliman. This is proving to be a Good Thing. Also last week we have identified and eliminated a major memory leak that was making OpenSim eventually run out of memory.

Hopefully when we get to 1.0 we will be able to consistently support 50+ avatars simultaneously under 500M of RAM and without crashing.

There is a new diva distribution available. It is packaged out of the bleeding edge OpenSim, revision 11056. If you have the previous release installed, you can simply run Update.exe.

In the past 2 weeks there has been a lot of plumbing in OpenSim, and things have improved considerably. First and foremost, we have identified and eliminated a memory leak that was causing OpenSim to use all memory over time, and eventually crash. Now OpenSim runs on much more reasonable memory footprint, and stays within that limit [for much longer].

Second, we have rewritten the grid service from scratch. The grid service is the part of OpenSim that manages region registration and lookup. All OpenSim installations have a grid service, even if they are standalones. In fact this distinction between standalone and “grid mode” is becoming fuzzier and fuzzier. So much so that it is now possible to have grids by stitching together standalone installations of OpenSim — without having to run any other servers!

Here are the instructions for how to do it with the diva distro. Instructions on how to do it with stock OpenSim can be found elsewhere.

Instructions for setting up a serverless grid with the Diva distro:

1 - If you want the second instance on the same machine, copy the entire folder of your diva distro into another folder on that machine. If you want the second instance on another machine, copy it to that other machine. Leave the original unchanged. In the copy (or copies), do the following steps.

2 - Edit Regions/RegionConfig.ini, and change the following fields for each region: (a) the name; (b) the RegionUUID; (c) the Location; and eventually (d) the InternalPort. For example, in my case my first region in my original instance is:

[Diva’s World 1]
RegionUUID = “9ca5b0b5-0b1e-47b7-ac30-7e0230152de0″
Location = “6178,3371″
InternalPort = 9000

In the copy on the same machine I have:

[Diva’s World 5]
RegionUUID = “854a1e57-95f2-42f9-b3c7-2488e0e0bd92″
Location = “6176,3371″
InternalPort = 9004

A couple of important notes:
* Please use truly random UUIDs. Here is a site that generates them for you: http://www.guidgenerator.com/.
* In my case I wanted to place the new 2×2 megaregion to the West of my original megaregion. Hence the coordinates 6176,3371 on the first region of the copy — the X coordinate is 2 to the left, the Y coordinate is the same.
* Megaregions need to be specified like what they are in that file: the first region is the SW corner, the second is the NW corner, the third is the SE corner and the fourth is the NE corner. Make sure you preserve this order when you make the changes in their coordinates.
* The InternalPorts must be unique for each region, on each machine. In my case, my second instance is on the same machine, so I had to change it. If your second instance in on another machine you can use the same InternalPort numbers of the original megaregion.

Do the necessary changes for the other three regions on that file.

3 - Edit config-include/MyWorld.ini. The only thing you need to change here, if at all, is in the [Network] section, the http_listener_port. If your second instance is on the same machine, you must change this port — for example 9001; this port needs to be unique per instance, per machine. If your second instance is on another machine, you can leave it as is. In any case, leave the server urls as they are for the original instance.

Et voila!

Login as usual. You will notice that the SW corner of your original instance now has neighbours to the left. In principle, you can cross back and forth, and teleport between regions as normal. In practice, crossing between instances with megaregions is still under heavy work and may produce surprising behaviour. For example, crossing from the original SW corner to the copy SE corner may end up being an auto-teleport, or you may get stuck in limbo, etc. Don’t be alarmed, this is “normal” behaviour for the time being, and these quirks will be fixed in future releases. For the time being,whenever you want to switch instances I recommend teleporting between SW corners of the different instances — that works pretty reliably.

It’s no secret that I love OpenSim standalone installations more than “grids”. For most things I envision doing with virtual worlds, including installations in my university, I can’t see the value of having a huge virtual space that is both shared between several tenants and centrally managed. The things I like to do fit perfectly in a few regions managed and controlled by me. The UCI Grid is about to become a web of hypergrided standalones. I suspect that many people are on the same boat. It used to be the case that standalones were solitary worlds that didn’t connect to anything or anyone; with the Hypergrid, that has changed. Standalones now can connect to the rest of the Metaverse — OSGrid, ReactionGrid, your friend’s standalones, etc. It’s a big world out there, and you can access it from your own small world.

I’m happy to announce Metaverse Ink’s “Diva Distribution”– a lean and mean OpenSim distribution targeting hypergrided standalones that comes preconfigured and that is very easy to keep up to date with newer releases of OpenSim. The Diva distribution is, literally, my personal preferences using the many configuration options of OpenSim. Here is what it consists of:

• MySQL as the DB backend. I used SQLite for a long time. SQLite is really easy to use, it requires no installation, it’s great for beginners. However, it’s clear that SQLite doesn’t scale well for long-term worlds. Moreover, using a MySQL server on the backend enables very interesting hybrid architectures that I will be talking about soon. The diva distribution requires some expertise for setting up MySQL, but it’s worth it. The instructions that come on the distribution make the MySQL setup for OpenSim really easy.
• ODE Physics. Easy choice here. Even the official OpenSim distribution is about to change to ODE as the default physics engine.
•  XEngine as the scripting engine, and lsl as the scripting language. XEngine is the most stable of the scripting engines we currently have, and lsl is the most widely used scripting language. All other language support is still experimental and fairly insecure.
• One Megaregion of 512×512 meters. That’s right! Megaregions just made it into OpenSim, thanks to Teravus, and they are working great. With megaregions there are no border crossings, everything is very smooth.
• MetaverseInk Search. If you want to make your world searcheable, mark your parcels “Show in search”, and they will be listed on MISearch’s engine. Others can teleport to your world from search results. Everyone can do this on their OpenSim by setting a certain configuration variable; the diva distribution has that preconfigured.

Over time we plan to add more goodies to the diva distribution, so stay tuned.

Additionally, the diva distribution contains two tools that make it really easy to configure and update your installation. They are:

• Configure: this is the tool you should run after unziping the first distribution. Be ready to tell it (a) the name you want to call your world; (b) the password for the MySql DB opensim account; (3) your externally visible IP address or domain name. Once you answer these questions, Configure will do all necessary configurations for you. You don’t need to edit any .ini files.

• Update: once you get the first installation set up, updating it is as easy as running the Update tool. Update asks no questions from you, it just does the right thing.

While the diva distribution comes preconfigured with my own preferences, it is possible to change these preferences to yours. For example, if you want more than a 512×512 megaregion, you can add more regions. The instructions for how to do that come in the documentation that is included in the distribution.

So where can you get it? Get it at http://github.com/diva/diva-distribution/downloads — the file to download is called diva-rNNNN.zip, which is listed at the bottom of that page.
Unzip it, read README, and follow the instructions from there.

Let me finish this post by reminding everyone that the current version of the Hypergrid, which we call HG1, is fairly insecure. Don’t take your favorite avatar to places that you don’t trust. HG2 is coming, but it’s not here yet.

Follow me on Twitter!

This is a post-vacation resolution to give some balance to my inherently obsessive personality: I’ll spread the energy among many addictions! Small twits on Twitter, bigger twits here. All this to prevent me from falling into that mental zone where nothing matters but the software.

Well, well, well. I’ve been so busy, I haven’t had time to post anything. A lot has happened since my last post. First and foremost, I got so much into OpenSim that I ended up being invited to become a core committer, back in December. This threw me into a serious case of programming addiction, of that kind that you neglect everything else. It was great! My recent one-month vacation got me out of the addiction, let’s see for how long… But here’s what I got so excited about: the Hypergrid.

The Hypergrid (HG) is a system architecture supporting decentralized virtual worlds. In other words, you host your world and it can interoperate with worlds hosted by other people and organizations. Yes, it’s a Web of virtual worlds. Users can teleport seamlessly between worlds.

I did the Hypergrid back in October/November, and it got integrated in the main distribution of OpenSim since then. Lots of people have their sims HG-enabled, including in the main open grid OSGrid.

The work on the Hypergrid is not over yet, it’s only half-way there. One thing is to enable this kind of interoperability; another thing is to make it safe for everyone. Since March or so I have been working on that second side of the issue — security. It requires changes in the client, so it’s a bit more tricky to deploy these changes. But they’re coming.

I will keep making posts about the Hypergrid. In the meantime, here are some useful links:

The main technical reference / white paper

Wagner Au’s post

Another Wagner Au’s post

Interesting ‘Yellow Pages’ of HG sims

An on-line publication about OpenSim and the HG

… and if you want to find Hypergrided sims, simply use MI Search.