Metaverse Ink

There is a new diva distribution available. It is packaged out of the bleeding edge OpenSim, revision 11056. If you have the previous release installed, you can simply run Update.exe.

In the past 2 weeks there has been a lot of plumbing in OpenSim, and things have improved considerably. First and foremost, we have identified and eliminated a memory leak that was causing OpenSim to use all memory over time, and eventually crash. Now OpenSim runs on much more reasonable memory footprint, and stays within that limit [for much longer].

Second, we have rewritten the grid service from scratch. The grid service is the part of OpenSim that manages region registration and lookup. All OpenSim installations have a grid service, even if they are standalones. In fact this distinction between standalone and “grid mode” is becoming fuzzier and fuzzier. So much so that it is now possible to have grids by stitching together standalone installations of OpenSim — without having to run any other servers!

Here are the instructions for how to do it with the diva distro. Instructions on how to do it with stock OpenSim can be found elsewhere.

Instructions for setting up a serverless grid with the Diva distro:

1 - If you want the second instance on the same machine, copy the entire folder of your diva distro into another folder on that machine. If you want the second instance on another machine, copy it to that other machine. Leave the original unchanged. In the copy (or copies), do the following steps.

2 - Edit Regions/RegionConfig.ini, and change the following fields for each region: (a) the name; (b) the RegionUUID; (c) the Location; and eventually (d) the InternalPort. For example, in my case my first region in my original instance is:

[Diva’s World 1]
RegionUUID = “9ca5b0b5-0b1e-47b7-ac30-7e0230152de0″
Location = “6178,3371″
InternalPort = 9000

In the copy on the same machine I have:

[Diva’s World 5]
RegionUUID = “854a1e57-95f2-42f9-b3c7-2488e0e0bd92″
Location = “6176,3371″
InternalPort = 9004

A couple of important notes:
* Please use truly random UUIDs. Here is a site that generates them for you: http://www.guidgenerator.com/.
* In my case I wanted to place the new 2×2 megaregion to the West of my original megaregion. Hence the coordinates 6176,3371 on the first region of the copy — the X coordinate is 2 to the left, the Y coordinate is the same.
* Megaregions need to be specified like what they are in that file: the first region is the SW corner, the second is the NW corner, the third is the SE corner and the fourth is the NE corner. Make sure you preserve this order when you make the changes in their coordinates.
* The InternalPorts must be unique for each region, on each machine. In my case, my second instance is on the same machine, so I had to change it. If your second instance in on another machine you can use the same InternalPort numbers of the original megaregion.

Do the necessary changes for the other three regions on that file.

3 - Edit config-include/MyWorld.ini. The only thing you need to change here, if at all, is in the [Network] section, the http_listener_port. If your second instance is on the same machine, you must change this port — for example 9001; this port needs to be unique per instance, per machine. If your second instance is on another machine, you can leave it as is. In any case, leave the server urls as they are for the original instance.

Et voila!

Login as usual. You will notice that the SW corner of your original instance now has neighbours to the left. In principle, you can cross back and forth, and teleport between regions as normal. In practice, crossing between instances with megaregions is still under heavy work and may produce surprising behaviour. For example, crossing from the original SW corner to the copy SE corner may end up being an auto-teleport, or you may get stuck in limbo, etc. Don’t be alarmed, this is “normal” behaviour for the time being, and these quirks will be fixed in future releases. For the time being,whenever you want to switch instances I recommend teleporting between SW corners of the different instances — that works pretty reliably.

It’s no secret that I love OpenSim standalone installations more than “grids”. For most things I envision doing with virtual worlds, including installations in my university, I can’t see the value of having a huge virtual space that is both shared between several tenants and centrally managed. The things I like to do fit perfectly in a few regions managed and controlled by me. The UCI Grid is about to become a web of hypergrided standalones. I suspect that many people are on the same boat. It used to be the case that standalones were solitary worlds that didn’t connect to anything or anyone; with the Hypergrid, that has changed. Standalones now can connect to the rest of the Metaverse — OSGrid, ReactionGrid, your friend’s standalones, etc. It’s a big world out there, and you can access it from your own small world.

I’m happy to announce Metaverse Ink’s “Diva Distribution”– a lean and mean OpenSim distribution targeting hypergrided standalones that comes preconfigured and that is very easy to keep up to date with newer releases of OpenSim. The Diva distribution is, literally, my personal preferences using the many configuration options of OpenSim. Here is what it consists of:

• MySQL as the DB backend. I used SQLite for a long time. SQLite is really easy to use, it requires no installation, it’s great for beginners. However, it’s clear that SQLite doesn’t scale well for long-term worlds. Moreover, using a MySQL server on the backend enables very interesting hybrid architectures that I will be talking about soon. The diva distribution requires some expertise for setting up MySQL, but it’s worth it. The instructions that come on the distribution make the MySQL setup for OpenSim really easy.
• ODE Physics. Easy choice here. Even the official OpenSim distribution is about to change to ODE as the default physics engine.
•  XEngine as the scripting engine, and lsl as the scripting language. XEngine is the most stable of the scripting engines we currently have, and lsl is the most widely used scripting language. All other language support is still experimental and fairly insecure.
• One Megaregion of 512×512 meters. That’s right! Megaregions just made it into OpenSim, thanks to Teravus, and they are working great. With megaregions there are no border crossings, everything is very smooth.
• MetaverseInk Search. If you want to make your world searcheable, mark your parcels “Show in search”, and they will be listed on MISearch’s engine. Others can teleport to your world from search results. Everyone can do this on their OpenSim by setting a certain configuration variable; the diva distribution has that preconfigured.

Over time we plan to add more goodies to the diva distribution, so stay tuned.

Additionally, the diva distribution contains two tools that make it really easy to configure and update your installation. They are:

• Configure: this is the tool you should run after unziping the first distribution. Be ready to tell it (a) the name you want to call your world; (b) the password for the MySql DB opensim account; (3) your externally visible IP address or domain name. Once you answer these questions, Configure will do all necessary configurations for you. You don’t need to edit any .ini files.

• Update: once you get the first installation set up, updating it is as easy as running the Update tool. Update asks no questions from you, it just does the right thing.

While the diva distribution comes preconfigured with my own preferences, it is possible to change these preferences to yours. For example, if you want more than a 512×512 megaregion, you can add more regions. The instructions for how to do that come in the documentation that is included in the distribution.

So where can you get it? Get it at http://github.com/diva/d2/downloads — the file to download is called diva-rNNNN.zip, which is listed at the bottom of that page.
Unzip it, read README, and follow the instructions from there.

Let me finish this post by reminding everyone that the current version of the Hypergrid, which we call HG1, is fairly insecure. Don’t take your favorite avatar to places that you don’t trust. HG2 is coming, but it’s not here yet.

Follow me on Twitter!

This is a post-vacation resolution to give some balance to my inherently obsessive personality: I’ll spread the energy among many addictions! Small twits on Twitter, bigger twits here. All this to prevent me from falling into that mental zone where nothing matters but the software.

Well, well, well. I’ve been so busy, I haven’t had time to post anything. A lot has happened since my last post. First and foremost, I got so much into OpenSim that I ended up being invited to become a core committer, back in December. This threw me into a serious case of programming addiction, of that kind that you neglect everything else. It was great! My recent one-month vacation got me out of the addiction, let’s see for how long… But here’s what I got so excited about: the Hypergrid.

The Hypergrid (HG) is a system architecture supporting decentralized virtual worlds. In other words, you host your world and it can interoperate with worlds hosted by other people and organizations. Yes, it’s a Web of virtual worlds. Users can teleport seamlessly between worlds.

I did the Hypergrid back in October/November, and it got integrated in the main distribution of OpenSim since then. Lots of people have their sims HG-enabled, including in the main open grid OSGrid.

The work on the Hypergrid is not over yet, it’s only half-way there. One thing is to enable this kind of interoperability; another thing is to make it safe for everyone. Since March or so I have been working on that second side of the issue — security. It requires changes in the client, so it’s a bit more tricky to deploy these changes. But they’re coming.

I will keep making posts about the Hypergrid. In the meantime, here are some useful links:

The main technical reference / white paper

Wagner Au’s post

Another Wagner Au’s post

Interesting ‘Yellow Pages’ of HG sims

An on-line publication about OpenSim and the HG

… and if you want to find Hypergrided sims, simply use MI Search.

You knew this was coming, and here it is: Metaverse Ink Search is now serving OpenSims around the world! This has been in the works for a while — in fact, it has been working for a while — but we kept it under wraps. A month ago or so, Eric Reuters wrote an article about it, but we still didn’t say much of how this whole thing works for OpenSim. This post explains it.

We worked with the OpenSim project in implementing the basic plumbing for search engines to operate in virtual worlds. We think this basic plumbing is the right support for searching virtual worlds, be them closed or open to the world wide web, and we are happy to see the grid-wide search project now using this same basic plumbing. The fundamental design philosophy is that the region’s data exposure to clients other than the viewer is a decision that involves 3 authorities: grid operators (if regions are connected to grids), region administrators, and parcel owners/content producers:

• Grid operators decide whether the grid is to be closed or open, and the amount of openness.  For closed grids, their data should be off-limits to external search engines like MI Search, just like the data in Bank Of America is off-limits to Google. This can easily be achieved by setting region configuration variables appropriately. Those configuration variables can be set so that only a grid-wide search engine can access the regions’ data.
• For open grids and standalone sims, region administrators decide whether to expose the data, the level of data exposure, and *which search engines can access it*. This last part makes virtual world search subtly different from web search, and for the better. On the web, search engines find web servers, and their pages, by following hyperlinks. All data reached is assumed to be public and, as such, indexable. This has caused a lot of pain on the Web, so let’s not go there. The idea here is that region administrators decide who indexes their data, and how much data is exposed to them. Data can be exposed to one or more virtual world search engines.
• Finally, parcel owners and content producers decide what data is intended to be found by checking the “show in search” check box in parcels and objects. In OpenSims, that checkbox tends to be free of charge.

All this is accomplished by configuring a core module of OpenSim called DataSnapshot. The configuration is done in OpenSim.ini, in the section pertaining to the DataSnapshot module, and it allows the variety of behaviors described above. So if you want the searchable data in your region listed with MI Search, go ahead and configure OpenSim.ini for it.

A second optional module deals with serving images (see how it looks here). This module, called ImageService, is a region module that extends regions with the ability to serve images onto the Web, so that explicitly marked, rich 2D content can be shared within a grid and/or with the whole world. The only images served are those whose keys are exposed via the DataSnapshot module, i.e. images related to things marked for search. That way we avoid exposing the entire collection of textures stored on the grid/asset server. The ImageService module is available from OpenSim’s GForge. If you want MI Search to show pictures of your wonderful regions, go ahead and place the dll of the ImageService in your OpenSim/bin directory.

The DataSnapshot module has been part of OpenSim since April 2008; the ImageService is our latest addition. OpenSim region operators have been able to figure it out, even though there were no announcements whatsoever. So much so, that we now have hundreds of regions on our opensim index in a variety of places, grid-ed and standalones. We are really excited to provide the first glimpse of the emerging Virtual World Web!

Welcome to the very first official version of Metaverse Ink Search, formerly known as SLBrowser. If you use SLBrowser routinely, you probably noticed a few minor changes in the graphic design of the search results pages. Those small visual changes are the tip of the iceberg! Below it, we’ve made several major improvements, as well as a rebranding move. Let me talk about each one of those in separate, and what they mean to you, whether you are a searcher or a content provider or both.

Data Collection Control

Our default listing consists only of those parcels that are marked “Show in search”, and that Linden Lab places on the public web — since the information is on the web, it is public information; google indexes it, so do we. Additionally, we have a simple and fail-safe mechanism for additional opt-ins and opt-outs with our search engine. Anyone wanting to list their parcels and products with Metaverse Ink Search, but not wanting to pay LL the search fee, can do so for free and automatically simply by adding the avatar MetaverseInk Masters to the list of Allowed residents. Besides listing the products inworld, we provide scripts that people can use to list their vendor products with our search engine. Conversely, if people want to be listed with LL’s search by marking “Show in search” but don’t want to be listed with us in particular, even though they may be listed with Google, we honor that. They simply need to add MetaverseInk Masters to the list of Banned avatars. See listing and delisting for complete information.

This way, the data collection control and cost control are ultimately on people’s hands. We like that!

Use of Images

Virtual worlds are full of wonderful images that people attach to their products and places. These images give searchers a preview of what can be found in those virtual worlds. People can compare before deciding to visit some place. There is no question that images enhance the searchers’ experience. Linden Lab has placed all of the images on the web, where the world can see/grab/link-to them. They are all available under http://secondlife.com/app/image/<LLUUID>/<size>, where <LLUUID> is the identifier of the image and <size> is either 1, 2, or 3. Had Linden Lab not placed the images on the public Web, we would not be able to show them on our web interface. LL’s placing them on the Web is a clear indication that they want to share those images with the Web-at-large.

We understand that some content creators — most notably texture artists — have concerns about seeing their textures flat on a web browser. They are not happy with LL’s small watermark on the bottom right. We sympathize with their situation, so we provide a method for creators to delist their publicly available images from our search engine in particular. That doesn’t remove their images from the Web; it simply removes them from our search results.

Rebranding

As we’re launching 1.0 of our search engine with all these improvements, we’re also rebranding from SLBrowser to Metaverse Ink Search. This has been in the workings for some time, since Metaverse Ink is not just about search in Linden Lab’s grid. So instead of the question mark with dots on top that we used to have as logo, you’ll now see a feather and the letter M.

Good searches!

We’re delighted to announce that Metaverse Ink’s OpenSearch has been selected as the default interface for search in the new viewer realXtend, whose 0.2 release was announced today. The realXtend project has been working closely with OpenSim in order to add some major features to it, both in terms of graphics and in terms of the underlying architecture of virtual worlds. The most important addition is the idea of an “avatar system”, a piece of the architecture that sees avatars as entities that can move freely between worlds — not just sims on the same grid, but sims on different grids. An avatar has an identity and an inventory, and that’s what’s being served from that component. The avatars themselves can now be created with a standard 3D modeling tool.

We believe that OpenSim, with these realXtend additions and other additions that are already in the making, is the future of interoperable virtual worlds. We are excited to be a part of it!

So, download realXtend, and try it out. Don’t forget to click the Search button and see OpenSearch natively served. Now you have a choice on the search engine you use for each search, just like what happens in modern web browsers.

Happy Searches!

As Felix explained yesterday, thanks to DrenBoy, we found a way of hooking up any search engine into the newest release candidate, so we quickly were able to implement our vision of OpenSearch. (We have a JIRA request on that) That was working great… big window, lots of space to see the search results… up until 2 hours ago, when Linden Lab released a new release candidate. This newest one doesn’t allow the search window to resize. Why oh why, I wonder…

We can still run OpenSearch in it, but the window is smaller. It’s bigger than the Web-on-the-profile, but it can’t be made even bigger.

I’m starting to suspect that someone at Linden Lab is doing an intentional ban on having web-browsing capabilities within the viewer. For example, the Web-on-the-profile is a teaser; like “we can do it but we really don’t want to.” Same thing with the search window. Even though what we see from Linden’s search is HTML, they go to great lengths to make it look like it isn’t, giving it a look&feel of all the other non-html windows.

OK, it’s their world, they can do whatever they want. In particular they can prevent a whole pandora’s box of competing emergent behavior related to the use of Web browsing inside SL. I can only say “tss tss tss” while shaking my head in disapproval. This is definitely not the openness that they talk about in their PR.

It’s pretty clear to me that any 3D world that wants to be open, like the Web, but immersive and social, like real life, must incorporate the Web and add the 3rd dimension to it. There’s no point in reinventing the wheel for the necessary 2D content that the human race has come to adopt in many flavors for storing and transmitting information. But maybe Linden Lab doesn’t want Second life to be the 3D Web after all, that was just a temporary PR glitch, or maybe a collective misunderstanding. Looks like Linden Lab wants Second life to be their own 3D game. They have all the right in the world to do that.

My only hope for this is that the general idea of the Metaverse has been placed out in the open. Sooner or later it will happen, with or without Linden Lab. The real OpenSearch will probably have to wait until that happens.

Forget the “profile web page” hack we tried… that’s sooooo last month. Now we can truly integrate directly with the built-in search. Here’s the scoop:

Linden Labs included a new search feature in the Windlight release candidate client of Second Life. At first it looked like another closed system, but then DrenBoy Opus discovered that the search page is configurable via an obscure setting. Since Diva Canto and I have been running our SLBrowser search engine for the last 6 months, we’ve been looking for better ways to hook into SL. We turned DrenBoy’s discovery into something useful by creating Open Search for Seocnd Life to augment the Linden Search with some search alternatives. The Open Search page gives you quick access to multiple search pages, including SLBrowser, Linden Labs, OnRez, SLExchange, and Google. The line-up is subject to change, but this is our current offering. You get all the functionality of Linden Labs new search, plus these other great search engines. We believe that search should be open, and that the best way to improve things is for competition… where the users choose the best solution, rather than being forced to go with once choice. If you have the new Release Candidate (see the link above) you can easily configure it for Open Search. Thanks to Bettina Tizzy and DrenBoy Opus for letting us know about this possibility.

The SL grid has been plagued by technical problems in the last week or so. The problems affect several functions, including LL’s search. But those problems don’t affect SLBrowser. Our search is always up even when LL’s is down, and even if you choose to access it from within SL using your web-on-the-profile.
Besides the obvious selling point here, there is a much more subtle, but important, architectural point that I want to make with this post. SLBrowser search is the living proof that several features of the metaverse not only don’t need to be centralized, but benefit tremendously from being provided by independent groups/companies. Search is one of those features, but there are a lot more. The good old principle of separation of concerns has served hardware and software design well for many decades.

As the community moves forward with virtual worlds, including efforts to design the next generation architecture of an SL-like scalable system, it is important to keep this principle mind.