Some time back, Melanie added a new flag to items, the “Export” flag, which some viewers started to support. The Export flag allows creators of content to mark them as exportable or not exportable — meaning ok to export to other grids or not. That gives some level of flexibility to content creators, but it doesn’t solve the problem. If an item is marked “No copy” but “Exportable”, a bad grid operator can easily change the permissions on it. So, at the moment, our recommendation to viewer developers is that they only allow to mark the “Export” flag when the item is fully “free”: no constraints on copy, transfers, and modifications. This is so people don’t get a wrong sense of security.

I should note that there are no reports of Hypergrid thefts, even though they are technically possible. As far as I know, most copy-botted content comes from using altered viewers on closed, non-HG grids; those loots are then imported into other OpenSim-based grids via those viewers too. So altered viewers, not HG thefts, are the main problem. Nevertheless, the fact that the HG opens additional doors for abuse has been a deterrent for commerce on the Hypergrid, so much so that we recommend “Export” only for freebies. It would be nice to change that perception. The incomplete proposal I have here solves the problem not just for the Hypergrid but also for viewer-based thefts in closed grids, even if those grids don’t ever want to connect to the Hypergrid. It solves the problem so thoroughly that, if implemented, it would allow commerce on the Hypergrid to bloom. The problem is in the implementation; hence, the “incomplete” in the title of this post…

The proposal, in a nutshell: ledgers.
Specifically for the Hypergrid: distributed, decentralized ledgers. Yes, blockchain, but minus the coin.

How it works, birds-eye view

Let’s ignore implementation details for a moment. I’ll cover them at the end.

Here is the main idea of a ledger: every time an item changes hands (from one avatar to another) a record about the transaction should be entered on a database. The information should include: the item ID, the time of original creation of the item, the timestamp of the transaction, the owner ID, the receiver ID, and the item’s permissions at the time of the transaction. Ledgers are append-only databases: they hold the historical record of all transactions since the beginning of time. (A side note to mention that OpenSim already has append-only databases: the assets.) So if any item’s origin is ever in question, the ledger can be queried for the ground truth.

Next, I will go through various theft scenarios, and explain how the ledger gives honest grid operators the technical justification for punitive action against thieves, and how it it will help identify (and, hopefully, socially isolate) dishonest grid operators.

Detecting theft within a closed grid

Let’s ignore the HG for a moment, and focus on centralized ledgers.

Let’s consider the viewer-based theft scenario in closed grids: some bad actor logs in with an altered viewer and copies all content that it sees into this bad actor’s inventory, changing permissions, and even authorship, along the way. This bad actor goes on to sell this content as their own on that same grid. Nothing in this proposal prevents this from happening. However, if and when the original creator detects this shenanigan, they can file a complain with the grid, who now is armed with the ledger, the historical record of all transactions. A simple query on the ledger can find out when the items in question were first transactioned; the offender’s item’s timestamp will be later than the original creator’s item’s timestamp. If the items are, indeed, similar, as per the original creator’s complaint, then the timestamps, and the lack of a transaction chain between the items, are proof that there was a theft, and who the offender was. The grid owner can act swiftly.

Detecting theft across grids

Let’s continue to ignore HG for a little longer.

Let’s consider the same viewer-based theft scenario, but now the bad actor imports the stolen content from closed grid X into another closed grid Y via the altered viewer. Assume both grids have their own separate ledgers, and both grid operators are honest. Again, this proposal doesn’t prevent theft from happening. It’s only when suspicion of theft is raised that the matter can be investigated.

In this scenario, suppose the bad actor with the altered viewer imports the stolen content into grid Y as their own. Grid Y will store the item as if it was legitimately created by this person at time T1, when it was imported there. It knows nothing about its true origin. Some time later, there is a complain from the original creator who first created this item in grid X at time T0 < T1. Both grids can easily investigate by agreeing that the items are, indeed, very similar, and by checking their own ledgers independently. If they find that T0 < T1, then it’s clear theft was at play, and who the thief was. They can act.

Detecting theft between a closed grid and OSGrid

Continuing to ignore HG, let’s focus on hybrids like OSG: in OSG-like grids, many independent people control their regions, but they are all connected to central grid services operated by osgrid.org, specifically inventory; this ledger would be another grid-wide service. Assume the region operators may be bad actors, but that the grid services operators are honest.

In this scenario, a bad actor, armed with an altered viewer, logs in to a closed commercial grid, steals content via the viewer, and then imports it to OSG using their own region, where they not only have God-like powers but can also alter the simulator code. They have a lot more opportunities to tamper with the timestamps of creation, ownership, etc. But that doesn’t matter much, as long as inventory and ledger are centralized services of the grid, and are capable of correcting any region-sent information. When suspicion of theft is raised, both grids can check their ledgers for timestamps.

Managing transactions across the Hypergrid

Having separate ledgers is fine for grids that trust each other, but in the Hypergrid, some grid operators may be bad actors themselves, and they control their grid’s ledgers. If they steal something, they can then enter those items on their own databases as having been created in a date that is sufficiently in the past to escape proof of theft, or, even worse, to harass the original content creators! “I created this before you did!” That’s bad.

For this to work on the Hypergrid, we need to establish a distributed, decentralized ledger system, blockchain style. Here is how it works.

Every time an item changes hands (from one avatar to another) a record about the transaction should be entered onto the distributed ledger. The distributed ledger is a kind of replicated database over a peer-to-peer network, but where peers can independently verify the veracity of the information that is sent to them — sometimes directly and sometimes via some voting algorithm. If we trust the information on the distributed, replicated ledger, rather than on any single node, then we can always verify whether something was stolen or not across grids that are part of the network.

The weak link of decentralized ledgers is false information. How can we prevent false information from entering the ledger? Here are a couple of scenarios of false information:

• Problem: a bad grid operator creates a false record of a transaction between a visiting avatar and a local avatar. That is, they send to the ledger “HG visitor A gave a copy of item H to local avatar B at time T1″ when that transaction never happened.
  Solution: any transactions between avatars should always take place on the grid of origin of the avatar that gives the item. In other words, visitors can get things, but should not give things. All transactions should be local to the originator avatar. This can easily be checked at ledger-entry time by verifying that the issuer grid is the grid of the originator avatar. If it isn’t, nodes of the distributed ledger won’t accept the transaction.
  (We can implement non-local transactions, too, but that will complicate the giving procedure substantially, as it will require confirmation from the originator when it comes back to their grid)
• Problem: a bad grid operator changes the items’ creation time to the past. For example, they import stolen content with an altered viewer, and then they send the following record to the ledger “Item H was created in 12/12/2012.”
  Solution: ledger nodes need to verify that the timestamps of a creation record are credible.

Implementation Issues: Where Things May Come to a Halt

While the general idea will definitely be the best solution for protecting content, implementing it in practice for the Hypergrid is not a trivial matter.

Ledgers are append-only databases; their size can grow very large. This is a common problem of all blockchain solutions. For example, the Bitcoin ledger currently 275G, and it will continue to increase forever. In our case, we could either do some garbage collection when items are removed from inventory, or we could do what blockchain operators do: archive old records into separate, slower, databases. But these solutions aren’t very good for the Hypergrid, where a large number of single person grids exist, many of them running in people’s home networks. Those small shops would end up having to deal with a very large database, most of which is irrelevant to them; they would get it just because they would be part of this ledger network.

We can estimate how large a distributed ledger for the HG would be, and how fast it would grow based on the current estimates of people’s inventories. One thing that will definitely contribute to the large size of the ledger is if we enter every inventory item on the ledger: if anything created in inventory, whether it will be passed around or not, is entered into the ledger, the ledger will become unsustainably large very quickly. For reference, there are hundreds of millions of entries in OSGrid’s inventory service, and probably billions of items have been created there. Multiply that by 1000 grids. This would quickly grow to be a terabyte-size database.

We could focus only on the items that are transferred from one avatar to another, and use the timestamp of the first transaction as a proxy for time of creation. This approximation will fail to detect thefts if the item is stolen before the first transaction happens, and then the thief quickly makes a first transaction themselves. This should be rare, and people could be aware and compensate for it. It will definitely decrease the size of the ledger by several orders of magnitude, as the vast majority of items in people’s inventories aren’t passed to another avatar.

Still, even with optimizations, the ledger will be large, and this will be a problem for small standalone grids.

Perhaps the right thing to do is to decouple the ledger from the grids. We can implement a distributed ledger system that is operated by a few trusted people in the community, with access to enough resources, and that ledger can hold the transactions of everyone who wants to ensure the use of legitimate content in their grids. Any grid not on this ledger will be suspicious; social pressure will make them look bad.

Of course, nothing prevents a single entity from creating a centralized ledger for many independent grids, as a service. That will require complete trust in the ledger operator.

Anyway, I hope this post will trigger some discussion and maybe even some action, as it is clear that without a mechanism for content protection, commerce on the Hypergrid will never take off, and the nice professionally-created items will not be available to those with good intentions, such as Universities.

As promised at OSCC’15, here is a post explaining how to configure OpenSim in order to experiment with the OnLook viewer.

OnLook experiments with two orthogonal UX features: (1) the user’s representation inworld; and (2) the GUI. I’ll talk about them separately.

Camera-Only Mode

Description: The camera-only mode, as the name suggests, presents a view of the virtual environment without the user’s self-avatar. In other words, it’s a just a camera into the world. You can move the camera using the arrow keys, more or less like you would move the avatar. In reality, the user avatar is in the world, and others will see it; but it’s just standing there, without moving, and the person operating that avatar won’t see him/herself. (This could potentially be expanded to also remove other avatars except, perhaps, NPCs, but I haven’t done that yet)

Implementation: The server-side implementation of this mode is under OpenSim/Region/OptionalModules/ViewerSupport/CameraOnlyModeModule.cs

Configuration: Add this somewhere in your ini files:

[CameraOnlyModeModule]
     enabled = true
     ;; Users with UserLevel equal or less than this will be sent this mode.
     ;; Others above will not. 
     UserLevel = 0

Special UI Module

Description: The special UI module allows us to control the buttons shown on the bottom of the viewer. It also hides the top menu bar.

Implementation: the server-side implementation of this mode is under OpenSim/Region/OptionalModules/ViewerSupport/SpecialUIModule.cs

Configuration: There are two parts to the configuration. First you need to define the buttons you want to show in an XML file that is sent to the viewer when the user connects to your sim, then you need to turn the mode on.

I have an XML file that you can use as a starting point: get it from here and place it under your OpenSim bin/ViewerSupport. You can then do your own research and experimentation, comparing that file with the default one, for purposes of defining what buttons you want to show to, and hide from, your users.

To turn this on, add this somewhere in your ini files:

[SpecialUIModule]
    enabled = true
    ;; Users with UserLevel equal or less than this will be sent this mode.
    ;; Others above will not. 
    UserLevel = 0

Final Remarks

I want to stress that the purpose of this is NOT to design any one alternative UX/UI, so please don’t comment on the set of buttons that I chose to hide in the example XML file for the bottom panel. That’s besides the point. The point here is that YOU, as operator of an OpenSim environment, can decide which buttons to show and which buttons hide when users visit your world. And people will get different UX/UI as they visit different worlds.

Finally, OnLook is not a separate viewer project; it’s a proof-concept on configurable UX/UI using Singularity. It served to guide me through the study of how easy/hard it will be to do it for real with any existing SL-derived viewer.

Existing Documentation

• Coding standards.
  Summary: it’s the C# coding standards plus a couple of extra rules
• Code base overview. (credit: Justin)
  A lot more could be said, but documenting the framework properly is a huge amount of work. More below.
• Submitting patches.
  Summary: small is beautiful. I highlight the following paragraph:
  “Please put only one logical change in a patch at a time. Patches that contain more than one logical change tend to be larger, more complex and hence take more time to be applied. At worse, developers will tend not to look at them because it’s hard to disentangle all the possible effects. Multiple logical changes can be in a patch if they only affect a single feature (like, for example, the feature the patch enables).“

Major Acceptance/Rejection Criteria

The pointers above give the easy reasons for why patches may be rejected. For example, patches that don’t follow the C# coding conventions or that are trying to do many things at the same time will likely be either ignored, rejected or pushed back to the contributors with requests for improvements. But these are easy-to-spot problems. There are other problems that aren’t so easy to spot, but that are even more important: whether or not a patch follows the architectural principles of the project.

This may be a foreign concept to to many, so here is an introduction to software architecture. There are many definitions for what software architecture is. I like Dick Taylor‘s definition:

“A system’s architecture is the set of principal design decisions made during its development and any subsequent evolution”

Given this definition, software architecture is usually implicit in the code and present only in the minds of the developers, unless an explicit effort is made to externalize it, which is very rare in open source projects. That’s why it’s so critical that people who want to make non-trivial contributions to open source projects like OpenSimulator join the developers in their day-to-day conversations in order understand the dynamics of design decisions. Alternatively, they may develop a working relationship with one or more developers. In the case of OpenSimulator, most of the action happens in the IRC (irc.freenode.net #opensim-dev); all core developers are there most of the time.

Before explaining the high-level design decisions of OpenSimulator, let me continue to be academic for a moment and point to a wonderful effort made by a colleague in Delft with the goal of (1) teaching students about software architecture and (2) contribute to open source projects in the process. Please follow this link (it will open in another tab):

Delft Students On Software Architecture 2015

As you can see, it took teams of 3-4 students 10 weeks in order to document each of the projects. I won’t do it in one post. But I can explain the very high-level architectural themes of OpenSimulator:

• The core developers are the major stakeholders. Each one of us has high stakes in the code base. We are not the only stakeholders, obviously, there are many more. The large community of users also has a stake in the project; but they don’t have as much power as the core developers or even as the non-core developers. In OpenSimulator, people who contribute code have more power over its evolution than those who don’t. See for example the Power/Interest grid of Jekyll; OpenSimulator has a similar grid.
• The project is a *framework* packaged together with a default application, SL-ish. It’s really important to understand what software frameworks are, with their hot and frozen spots. Two consequences come from OpenSimulator being a framework, rather than a regular application:
  • Patches that touch frozen spots unnecessarily are likely to be rejected. OpenSim.Region.Framework, OpenSim.Framework.Servers, OpenSim.Server, OpenSim.Services.Interfaces, OpenSim.ApplicationPlugins.RegionModulesController and OpenSim itself are the major frozen spots. Everything else is a hot spot.
  • We encourage the encapsulation of new features in region modules (the plugins of the simulator), which extend/replace hot spots. If that implementation proves difficult, we’d like to know why it is difficult; we’ll gladly change the APIs when they prove to be getting in the way of doing things, but it requires discussion.
• We may or may not take in new plugins into the core distribution, depending on how important we think they are for the stakeholders. When we don’t take them in, we encourage their developers to maintain them and make them available to everyone, either by hosting the code in some open source repository or by distributing binary versions of their plugin.
• The existing plugins in the core distribution reflect a certain level of performance, operations and security that the core developers deem acceptable wrt usability and vulnerabilities. Alternatives to these implementations may change levels of performance and security (up or down), and sometimes those changes have tradeoffs in terms of usability and vulnerability. We may or may not accept those new tradeoffs in core. Patches that increase performance and security without any changes to usability and no additional vulnerabilities are usually very welcome and accepted immediately. Patches that move the levels of usability and exposure to vulnerabilities need a serious assessment. If we deem them unacceptable, we continue to encourage the developers to maintain their own implementations and even make them available to others; we just don’t want to have them in the core distribution.
• We encourage the community to develop and maintain their own plugins. Several core developers have their own alternative implementations of many of the core plugins in their own usage of OpenSimulator, as well as lots of additional features developed as plugins. That’s the whole point of having a framework. The core distribution reflects a certain balance in a large spectrum of implementation options that we believe benefit a large portion of the stakeholders, but possibly not all.

Examples of Accepted and Rejected Patches

Here are some examples of patches that have been recently accepted and rejected. For each, I give a brief reason for the decision:

• 7254. Status: accepted. Reason: very small patch, harmless change to existing Robust plugin.
• 7673. Status: accepted. Reason: small bug fix patch to frozen spot, confirmation of resolution by users.
• 7682. Status: accepted. Reason: very small patch, harmless bug fix to Linden client plugin.
• 7345: Status: accepted after discussion. Reason: very small patch for fixing annoying bug in AttachmentsModule plugin. Issue of discussion: hard-coded sleeps and placement of the code.
• 7676. Status: rejected.  Reason: very small patch, but with unacceptable downtime for existing grids.
• 7677. Status: rejected. Reason: very small patch, but with unacceptable change in security of one of the Hypergrid plugins vs. usability.
• 7653. Status: rejected as is, waiting for improvement. Reason: very large patch trying to do too many things at the same time, and with unnecessary intrusion in frozen spots of the framework. Part of it should be rewritten as a plugin, other parts could also be rewritten as a (different) plugin.

NOTE: mono addins works in both .NET and mono. If you run Wifi in Windows you do not need to install mono in order to run mono addins. Everything you need is already included in the OpenSim distribution.

Wifi for Grids

Let me recap how Wifi has been deployed in grids up to now. Again, operators of D2 standalones don’t need to know any of this, since Wifi comes pre-installed in D2.

Up to now, every time I made a new D2 release, I also made a special package (a zip file) with just the Wifi DLLs (see here). Some people in the community have written detailed instructions on what to do with these DLLs, which boils down to copying files manually to certain folders and then adding a few configuration sections to the Robust configuration file. (Wifi for grids runs in a Robust server)

From here on, I will stop producing this extra zip file. Instead, I am producing a mono addin package (which happens to also be a zip file, but with more information inside), and I’m making it available from the metaversink mono addins repository listed here. Installation and future updates are done via the generic mono addins mechanism, not by copying files by hand.

So here are the detailed instructions for how to install Wifi in your grid when updating from OpenSim 0.8.0 to OpenSim 0.8.1. Note: If you are making a new install from scratch, skip to step 6.

1. Make a backup of your WifiPages folder to somewhere safe, especially if you customized some of them. Likely you won’t need this backup, but better safe than sorry.
2. Update your robust installation to 0.8.1
3. Make sure to incorporate any OpenSim configuration changes in your Robust configuration file
4. (From here on, all console commands are supposed to be given from the opensim bin folder)
5. Edit your Robust configuration file and remove the DivaWifi server connector. Depending on how old your config file is, this may be at the end of a very long string called ServiceConnectors under [Startup] or in a variable under [ServiceList]. Wherever it is, remove it, because Wifi now will run as a mono addin.
6. Still in your Robust configuration file, go to the [Startup] section. Set the RegistryLocation and the ConfigDirectory to folders outside of the bin folder. For example:

    [Startup]
       RegistryLocation = "../../addins-registry"
       ConfigDirectory = "../../addins-config"

   Alternatively, you can give absolute paths, for example:

    [Startup]
       RegistryLocation = "/home/opensim/addins-registry"
       ConfigDirectory = "/home/opensim/addins-config"

7. Still in your Robust config file, go to the [DatabaseService] section. If you had

   StorageProvider = "Diva.Data.MySQL.dll"

   switch it back to the default

   StorageProvider = "OpenSim.Data.MySQL.dll"

   Don’t worry, Diva Wifi will use the right one, but the rest of your Robust will use the default.
   Save and close your Robust configuration file.

8. Populate the mono addins registry with all the OpenSim core addins:

    $ [mono] mautil.exe -reg /path/to/registry reg-update

   where /path/to/registry is the RegistryLocation you set in the configuration file, so for example ../../addins-registry or /home/opensim/addins-registry. IT’S VERY IMPORTANT THAT THIS PATH IS EXACTLY THE SAME AS THE ONE YOU WROTE IN YOUR CONFIG FILE FOR RegistryLocation.

9. Add the metaversink addins repo as one of your sources of addins, and update its contents locally:

   $ [mono] mautil.exe -reg /path/to/registry rep-add http://metaverseink.com/repo
   $ [mono] mautil.exe -reg /path/to/registry rep-update

10. You can now see the list of all the available addins:

    $ [mono] mautil.exe -reg /path/to/registry list-av

11. Install the Diva.Wifi addin:

    $ [mono] mautil.exe -reg /path/to/registry install Diva.Wifi

Note that this command, as is, will install the latest version of  Diva.Wifi. This may or may not be what you need. Typically, several versions of it will be available from the Metaverseink repository (e.g. for OpenSim 0.8.1, 0.8.2, etc.), and you can see them all when you list what’s available (with the command in step 10). So, if your OpenSim installation is not the latest, you  need to install the particular version of Diva.Wifi that corresponds to your OpenSim version.

At this point, you can run your Robust server:

$ [mono] Robust.exe -inifile YourRobustConfig.ini

As you will see, it will warn you that Wifi needs to be configured, something like this:

2015-04-04 14:53:54,273 ERROR – Diva.Wifi.WifiServerConnector [Wifi]: PLEASE EDIT ../../addins-config/Wifi.ini BEFORE RUNNING THIS SERVICE

If you don’t see this warning message, something is wrong, likely the path to registry is inconsistent. Recheck the previous steps starting in step 8.

What does this warning message mean? Up to now, the Wifi configuration section was included into your Robust configuration file; from here on, it is in a separate file. That file is placed under ConfigDirectory/Wifi.ini, where ConfigDirectory is whatever you set in your Robust config file (see point 5 above). So, for example, ../../addins-config/Wifi.ini or /home/opensim/addins-config/Wifi.ini.

So edit that file and set up everything appropriately as you had before (copy-paste from your old file will do it). The configuration variables themselves didn’t change. Only one was added: don’t forget to set Enabled = true.

Once you’re done, save the file and restart Robust. This time everything should work.

IMPORTANT: from here on you can delete the [WifiService] section in your main Robust configuration file, because it will not be used anymore. Wifi configs will be read from /path/to/addins-config/Wifi.ini

Updating Wifi in the Future

I know changes of process are always a bit painful, and you may be wondering why I changed this. The reason is twofold: (1) much easier updates from here on; and (2) additional optional goodies coming up. Let me explain the updates.

When you follow the instructions above, your installation of Wifi will be relatively independent of your installation of OpenSim. The Wifi app itself is in /path/to/registry/addins/Diva.Wifi.0.8.1.0.3/ and its configuration file in /path/to/config/Wifi.ini.

The next time that I make a new release of Wifi, all you have to do is to update Wifi, something like this:

$ [mono] mautil.exe -reg /path/to/registry rep-update 
$ [mono] mautil.exe -reg /path/to/registry list-update 
$ [mono] mautil.exe -reg /path/to/registry update Diva.Wifi

Voila! No copying things by hand anywhere. I’ll make sure to repeat these instructions again next time I make a release of Wifi.

Optional Diva Addons

The second reason why I did this is to be able to provide additional functionality in a modular manner. Take another look at the available addins:

$ [mono] mautil.exe -reg /path/to/registry list-av
Available add-ins:
 - Diva.AddinExample,0.8.1.0.1 (metaverseink.com)
 - Diva.AddinExample,0.8.1.0.2 (metaverseink.com)
 - Diva.Interfaces,0.8.1.0.1 (metaverseink.com)
 - Diva.MISearchModules,0.8.1.0.2 (metaverseink.com)
 - Diva.Modules,0.8.1.0.3 (metaverseink.com)
 - Diva.TOS,0.8.1.0.1 (metaverseink.com)
 - Diva.VirtualOffice,0.8.1.0.1 (metaverseink.com)
 - Diva.VirtualOffice,0.8.1.0.2 (metaverseink.com)
 - Diva.Wifi,0.8.1.0.2 (metaverseink.com)
 - Diva.Wifi,0.8.1.0.3 (metaverseink.com)
 - Diva.Wifi.GridManagement,0.8.1.0.1 (metaverseink.com)

What is all this? These are all Diva addons — modular functionality that I have been working on for some of my grids — several versions of them. Some of these addons are for Wifi, others are for the simulators. Also, some of them are free, others require a license. Here is an explanation of what they all are:

• Diva.AddinExample is just an example of how to write addins for OpenSim. It’s intended for OpenSim developers. An explanation can be found here. It’s free, but you really don’t want to install it, it’s a dummy addon.
• Diva.Interfaces is a generic piece of code upon which many Diva addons depend. It doesn’t provide any functionality by itself. Its free, but it doesn’t do anything by itself.
• Diva.MISearchModules contains functionality that people may want to install on their simulators, so that their content and images are listed in the MISearch engine. It’s included in D2, and it’s free.
• Diva.TOS contains functionality to support a Terms of Service acceptance step when people login or HG-teleport to a grid. It pertains to simulators. The Wifi part is already included in the free Wifi app, and this addon is included in D2. This addon is also free.
• Diva.VirtualOffice contains a simulator-bound feature that supports the upload and display of PDF files inworld. It requires a license.
• Diva.Wifi is…Diva Wifi. Free.
• Diva.Wifi.GridManagement is an addon to Wifi that provides a much richer set of features for grid management, such as estate and group management; it requires a license.

Up to now it was really painful to deploy this additional functionality on my grids, and pretty much impractical to make it available to others. This new mechanism makes it much simpler for me, and makes it possible for me to share it. Let me explain what you need to do if you want to install one of them, for example, Diva.MISearchModules.

1. Go to your opensim installation — the simulators, not robust. Change to the bin folder.
2. If you have a D2 installation, you can skip this step. If you don’t have a D2 installation, edit your configuration file OpenSim.ini, go to the section [Startup]. Set the variables RegistryLocation and ConfigDirectory to outside the OpenSim installation, for example:

   [Startup]
       RegistryLocation = "/home/opensim/addins-registry"
       ConfigDirectory = "/home/opensim/addins-config"

3. If your simulators installation folder is the same as your robust installation, you can skip this step. Otherwise do this to get the metaverseink repo and cache its contents locally:

   $ [mono] mautil.exe -reg /path/to/registry rep-add http://metaverseink.com/repo
   $ [mono] mautil.exe -reg /path/to/registry rep-update

   where path/to/registry is whatever you set it in RegistryLocation in step 1. In D2s it is ../../addins-registry

4. Install the addon:

   $ [mono] mautil.exe -reg /path/to/registry install Diva.MISearchModules

5. Reboot your simulator(s). You will likely get warnings about having to configure these addons. Follow the instructions in the warnings, and reboot the simulators again.
6. README files, if any, are placed under /path/to/registry/addins/<addin>

If you are interested in any of the non-free addons, let me know.

As I’ve announced at OSCC’14, I am working with Liru on a Singularity-based viewer whose main goal is to make the viewer’s UI as programmable as possible server-side. You know, more like Web browsers.

The idea here is to allow OpenSim operators to have more control over the experience that their users will have; specifically, I want to be able to design much simpler user interfaces that are more appropriate for people who have no experience with Second Life and/or may be even uncomfortable with seeing themselves as avatars. But more importantly that any specific UI, I want to be able to provide those UI specifications dynamically as the user enters the simulator, rather than being hard-coded during the viewer’s build process. This way, we can change the UI without forcing users to install new versions of the viewer. And no one needs to agree on any specific UI.

I’m happy to announce that the very first alpha release of OnLook is now available for testing! Without the server-side UI specs, OnLook behaves exactly like Singularity, so you won’t see any difference when you login to your world. In order to help people test it, I have deployed a small test world that can be accessed via the Hypergrid. So here is a description of the steps that you can do to test this, along with pictures of what you’ll see.

Step 1. Install OnLook

Here is the link to the installer in Windows and the link to the dmg for Mac. (Thanks to Cinder Biscuits for the Mac version)

Step 2. Login to your HG-enabled world

Login to your world somewhere. It needs to be HG-enabled. Once you login, the interface will be exactly like Singularity.

Step 3. Teleport to my Test world

Pull up the map and search for http://ucigrid01.nacs.uci.edu:8002. Teleport there.

Once you land, notice the new UI: The top menu disappears, and the bottom toolbar changes to show only a few options. This is a simplified UI that I’m defining and sending from the server. Others may want to send a different UI.

Next, click on Destinations and choose Epic Castle:

Step 4. Teleport to Epic Castle

Click to teleport to Epic Castle, which is another region on this same test grid.

This region is configured in “camera-only” mode. What this means is that your avatar will be invisible. For this alpha release, you will still see your attachments, if you have them, but we are working to make them invisible too, to make it truly “camera-only”. (Since I wrote this post, this has been fixed, so now your avatar will be completely transparent, attachments and all)

Notice that the controls are completely different too. If you press the arrow keys, your avatar stays in the same place, but the camera moves. Shift+arrows gives you more movements of the camera. Shift+Esc brings the camera to its original position. In any case, the avatar’s position stays unchanged when in this mode.

The objects in the scene are still interactive. See that yellow sphere? You can click it.

This mode is something that I wanted to do for a long time in order to make these wonderful worlds accessible to people who don’t like to see themselves as avatars. Encitra is going to use this mode in urban planning applications, as these environments become more like the modeling tools that urban planners are familiar with.

Before you leave this region, you may want to play with the options under the Settings button.

Step 5. Go back

The camera-only mode is very restrictive; you can’t even use keyboard shortcuts. In order to go back, pull up the Destinations guide and choose Convention Center.

That will take you to the first region you visited on this test grid. Your avatar will be visible again.

Finally, press Cntrl+Shift+h to go back to your home world. Once you arrive, notice the full-fledged Singularity UI coming back.

That’s it! Thanks for testing!

Final Words

OnLook is available in Github and the server-side modules are already in OpenSim(Dev), undocumented for now. Once they stabilize, I promise I will write documentation for how to use them. One thing to know is that these UI modes can be defined on a user level basis, meaning that you can have builders login with the default UI, and regular users login with a much simpler/restrictive UI.

This work has been supported by Encitra.

OpenSimulator 0.8 brings one much requested feature, profiles, and a fantastic new feature, variable-sized regions (varregions for short), something we got from an offshoot of OpenSim with collaboration from viewer developers. Profiles and varregions both work out of the box for new installations of D2, so if you’re doing it from scratch, you don’t need to read this. But if you are upgrading your D2, there is some manual upgrading to do. Please read on!

Profiles

For profiles to work in D2, edit your existing bin/MyWorld.ini and add this section somewhere:

[UserProfiles] 
  ProfileServiceURL = "http://myworld.com:9000"   (replace with your domain name)

Then add this extra variable under [LoginService]:

[LoginService] 
  SRV_ProfileServerURI = "http://myworld.com:9000"  (replace with your domain name)

How to Change D2 from Mega to Varregion

During all this time, D2 has been packaged with a precursor of varregions called mega-regions. Mega-regions, however, failed to work well on many details like crossing borders or teleports to the middle of the area. Varregions involves improvements in the viewers, so users need to use a relatively recent viewer like Singularity, but they solve all of the problems that mega-regions had, and make the specification of the space much simpler too.

Starting in this release (r24886, OpenSim 0.8), new installations of D2 use varregions out of the box instead of mega-regions. The default size of the space is as before 512×512, but it can easily be changed. Because the old mega-regions are difficult to deal with, existing D2 worlds are not automatically updated to varregions, so they will continue to work using mega-regions if nothing is done. However, I strongly recommend D2 users to update their worlds immediately after updating D2.

Please note: upgrading to varregion will eliminate the existing parcels in all but the SW area; you will need to redo the parcels again. Although this is inconvenient, I highly recommend changing to varregions now, because support for mega-regions in OpenSim will eventually be dropped.

1. Update your D2 to diva-r24886

    $ [mono] Update.exe

2. Change to diva-r24886/bin and restart OpenSim as usual

    $ [mono] OpenSim.exe

   Note that this will run your world still as a mega-region. Everything should work as before.

3. Make oars of all of your regions. Assuming the default 4-cell megaregion configuration:

    # change region "<SW>"  (where SW is the name of the lower-left region)
    # save oar sw.oar
    # change region "<SE>"   (where SE is the name of the lower-right region)
    # save oar se.oar
    # change region "<NE>"  (where NE is the name of the upper-right region)
    # save oar ne.oar
    # change region "<NW>"   (where NW is the name of the upper-left region)
    # save oar nw.oar

4. Shutdown OpenSim:

    # shutdown

5. IMPORTANT: Make a copy of bin/RegionConfig.ini for backup, in case you need to go back to mega-regions.
6. Edit bin/RegionConfig.ini. Again, assuming the default 4-cell megaregion that came with D2, you should see 4 blocks of region specification there, one per region. Leave the first one, and delete, or comment (with semicolon ‘;’), the other three. On the first block that is left, add the following variables:

    SizeX = 512
    SizeY = 512

   So, your entire (single) block should look something like this:

    [My World 1]
    RegionUUID = "bd60a94c-bebe-bebe-96bc-26564f652111"
    Location = "2238,2550"
    InternalAddress = "0.0.0.0"
    InternalPort = 9000
    AllowAlternatePorts = False
    ExternalHostName = "myworld.com"
    SizeX = 512
    SizeY = 512

7. Restart OpenSim

    $ [mono] OpenSim.exe

   You will now have 1 large region that has all the content, but the terrain everywhere except the SW area is wrong. Don’t panic! You are now going to load the rest of the terrain.

8. Leave the SW corner as is, since it already has the right content. Load the other 3 oars with the corresponding displacement:

   # change region "<SW>"  (where SW is the name of the lower-left, and now only, region)
   # load oar se.oar --displacement "<256,0,0>" --merge --force-terrain 
   # load oar ne.oar --displacement "<256,256,0>" --merge --force-terrain 
   # load oar nw.oar --displacement "<0,256,0>" --merge --force-terrain

That’s it! You now have a D2 with a proper variable-sized region that will work much better than the mega-region we had before. From here on, saving OARs will involve only this one region, not four. This is a true single region, not many regions pretending to be one, like mega-regions.

If you had previously changed the default 2×2 mega-region of D2 by adding more regions in bin/RegionConfig.ini, you need to delete all but the first region in that file and set the size to whatever your space size is. You will also need to save all of the oars for all of your regions and then load them into the single varregion with the corresponding displacements.

Changing the Size of Your World

If you want you can increase the size of your region by changing SizeX and SizeY in bin/RegionConfig.ini, and then restarting OpenSim. Two constraints must be observed: (1) SizeX=SizeY (so, only square regions are allowed); and (2) the size needs to be a multiple of 256.

I don’t recommend regions larger than 1024×1024, though, because the terrain will be massive.

As all of you probably already know, we are organizing the first OpenSimulator Community Conference on the first full weekend of September, 7-8. This event has been in the making since, roughly, March. But it took us a couple of years before that to actually convince ourselves that it was worth doing it. Here are all the reasons why I think this event a great thing, and why everyone who uses OpenSimulator should be involved in some way.

Reason #1: Dramatic Improvements to OpenSimulator Performance

When we started planning this in March and we faced the prospect of a 200 people event, we went “omg!” Although we had never even tried, we knew that we could not host a 200 people event with OpenSimulator as it was in March — period. The maximum number of people we had ever attempted was 52, and in that test, lag started lurking in at 20, and never went away;we hit 52 people, but no one could move and chat was laggy as hell; the simulator eventually crashed. That test was back in 2009, too, a long time ago. Apart from Intel and its Distributed Scene Graph work, which takes a drastically different  architectural approach, none of us has ever paid much attention to core simulator performance.  It was clear that if we were going to attempt a 200 people event, we would need to invest a lot of our time in making OpenSimulator perform not just as well as the Second Life simulators, but… better.

Well, 4 months and 926 commits later, we have now taken the performance of OpenSimulator to levels that it has never seen before. In recent load tests, we have had as many as 117 avatars distributed over a 4-sim corner, with one of the sims usually hosting close to 60 avatars. After initial arrival of avatars, which is still a heavy load, the CPU usage on those simulators stays well under 100%. There’s very little to no chat lag and people can walk around without freezing neither the sim nor their own viewers. (Although we ask people to sit, because that’s the scenario for the conference)

A sim with 10 people standing around and chatting, which is a typical scenario for these virtual environments, now uses as little as 15% CPU — you can practically hear it yawn!

Reason #2: Dramatic Improvements to the Stability of Existing Features

Just to name three: the Hypergrid, teleports in general and Groups.

The Conference Center grid is hosted in a special-purpose server that is completely independent from any existing grid; it’s its own grid. Account creation on that grid is limited, by design. Except for conference staff, it doesn’t make much sense to host user accounts and their corresponding inventory in the Conference Center grid, because this is just a one-time event. It makes more sense for participants to come from their own persistent home worlds. Indeed, in all of the load tests that we have been making, the vast majority of avatars come from other grids — 33 grids so far! Consequently, a few bugs and poor performance in the Hypergrid had to be fixed…

In chasing bugs, we realized that what we were doing with teleports was not exactly in accordance to what the viewer was expecting. So we completely rewrote the Teleport protocol. The new protocol conforms with what the Linden grid does, as far as we can tell. As a consequence, teleports are now more solid than they ever were.

As for Groups (V2), even though I added this to OpenSimulator back in February, it had never been tested at scale. In using groups in the Conference Center grid, it was clear that there were some major issues in there that had been inherited from the Flotsam groups implementation. One of the major ones was in Group chat, which was very laggy and.. well, wrong. Group chat is now fixed. In the last load test we had 100+ people chatting (abusing) over the Load Test group with no additional CPU usage, no duplicate messages and generally no noticeable bugs. We also plan to make heavy use of groups for access control and permissions. Our smaller tests indicate that this can now be safely done.

Reason #3: Increased Synergy with Viewer Developers

This has, perhaps, been the best surprise of the process: we have had an increasing synergy with some of the viewer developers, specifically Singularity. So much so that we are planning to release a version of Singularity that is branded for the conference event. Some of the Singularity developers have been part of the load tests, and they have been extremely responsive to information and change requests that make the whole system work better. For example, thanks to lkalif, I was able to make the edit beams work in OpenSim (have you ever noticed that in OpenSim the edit beams were gone?). Another example, because of the new way that the viewers handle the map tiles, the map was useless after an Hypergrid Teleport; thanks also to lkalif, this has now been fixed in Singularity!

I see a whole new world of possibilities now…

Reason #4: New Features in D2 Wifi

Q: how would sponsors bring their content to the Conference Center grid, especially those whose grids aren’t Hypergrid-enabled?
A: add a feature to Wifi that allows users to upload IARs.

Done. It will be part of the next release of Wifi.

Reason #5:  Community Building

The OpenSimulator core developers are a bunch of radically independent nerds that have shown more talent to code than to do community management. That’s an unalterable fact. Luckily for this conference, we’ve engaged with AvaCon, a non-profit whose sole purpose is to help foster communities in the Metaverse. It’s been great to see the volunteers and presentation proposals coming in, managed by them! Now that I see it, I can’t wait to hear what people have been doing with OpenSim!

Reason #6: The Program!

Finally, conferences can be fun to attend. The program is shaping up to be very interesting, starting with Grady Booch’s keynote and ending with the breakout sessions and 3rd party social events.  The panels are interesting, the presentations I’ve seen being proposed are also interesting. As the program goes, the conference looks like a serious real world conference, as far as I can tell. (The final  program will be announced soon!)

I don’t think I ever attended a virtual conference like this before, so this is a first! If we are really able to pull this off in September, we might see more conferences going virtual instead of contributing to waste the ozone layer!

So here are some ways for you to get involved and help in this multi-purpose activity:

• We are still in need of inworld volunteers. Consider volunteering to help during the conference. You’ll get an inworld ticket, and those are limited!
• Don’t have much time to volunteer or cannot attend the conference? Consider being a sponsor, even if at the lowest level (US$50). While a virtual conference is 80x cheaper than a RL world one, it is not free of costs entirely. These sponsorships help defray the costs that we are incurring in.
• Come participate in the load tests. We have one every Tuesday at noon PST. There is also a special load test this Saturday, where we hope to get close to 200 avatars — the real deal. Be part of a major milestone in the history of OpenSim/Second Life.

D2 is meant to be a personal virtual world server, so some of these addons have limitations that reflect the spirit of a personal virtual world. For example, in D2 you can create at most 2 groups. I have an unlimited version of Groups, too, but that is meant for larger worlds.

Anyway, you can find all the information about these addons here. The Wifi web app has been extended with some minimal support for managing groups.

These addons are works in progress, and I will keep improving them. This is their first release to the public, so expect some hickups. Please report those issues here, but make sure to read the documentation first.

Have fun!

If you are updating your D2 in Linux from r20232-b to the latest, you will very likely encounter the following problem:

$ mono Update.exe
Using download link http://metaverseink.com/download/diva-r22043.zip
Your current release is diva-r20232-b. Available release is diva-r22043
New release available. Updating…
Downloading new release from http://metaverseink.com/download/diva-r22043.zip to /tmp/diva-r22043.zip. This may take a few minutes.
……………………………………………………………:-)
Extracting new release to /home/opensim
* Unable to unzip. Please try updating later. (Reason: Path is empty)
*********************************************************************
Your installation has been updated. The new release is in
>> /home/opensim/diva-r22043 <<
Your regions have been migrated.
Please read the RELEASENOTES to know what changed.
If you made changes to OpenSim.ini, please copy that file over
or change the new one.
*********************************************************************

<Press return to exit>

In other words, the Update tool fails to unzip the new package.

This is due to a bug in a library included in OpenSim 0.7.4, on which diva-r20232-b is based. The offending library is called Ionic.Zip.dll. Here is what you need to do in order to get past this issue.

1 – Download this version of that library and place it in your current d2 bin folder, diva-r20232-b/bin. You can overwrite  the one that’s there, or move it somewhere before placing the new version there. Note that some download tools (e.g. wget) don’t overwrite files and instead give them new names. Make sure the new version has the right name.

2 – Run the Update tool again.
$ mono Update.exe

Voila! Problem solved.

The latest D2 release has the correct version of Ionic.Zip.dll, so hopefully this won’t happen next time you need to update.

127.0.0.1 localhost
127.0.1.1 nile.ics.uci.edu nile

# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

Yesterday, I spent a lot of time harassing just about everyone in my University’s Networking Support department because I couldn’t get an OpenSim server to serve clients anywhere. We know that OpenSim networking is hard. But, damn it, I’m one of the core devs of OpenSim, and I have a pretty good idea of what’s involved in the protocols between the server and the client. I’ve had lots of OpenSim servers running successfully in all sorts of networks, including my University’s. Well, the problem turned out to be… Ubuntu!

The default installation of Ubuntu comes with a horrible hack in /etc/hosts:

127.0.1.1 nile.ics.uci.edu nile

Yes, this maps the domain name to an interface in localhost, for reasons explained here. As stated in the Debian Reference:

“This is really improper because system hostnames and domain names are two very different things; but there you have it.”

How does this impact OpenSim?

Upon client login (over TCP), OpenSim sends its UDP address to the client. In doing so, it needs to send the actual IP address, not the domain name — the client expects the IP address. Due to that entry in /etc/hosts, OpenSim was getting nile.ics.uci.edu translated into the localhost IP… Fail!

Fix: either remove/comment that line from /etc/hosts or replace it with the external IP address of that server, if it has one.

Conclusion: bad Ubuntu! (Debian may suffer from the same problem)