Monthly Archives: January 2010

The Future of OpenSim: Virtual Yellowstone National Park

Posted on January 25, 2010 by Shenlei | Comments Off

The following video was taken in ScienceSim grid in the Geography regions with terrain for the Yellowstone National Park loaded.

Comments Off

Posted in secondlife

A Glimpse of the Future of OpenSim: Virtual Yellowstone National Park in ScienceSim

Posted on January 24, 2010 by Shenlei | 8 Comments

OpenSim turned three years old yesterday. In the past three years, we’ve been watching as the initial germ of an idea evolved and changed into the current platform, which is poised to go even further.

Fashion Research Institute has a unique view of the evolution of this open source virtual world platform. We are designers; users and consumers of OpenSim. We have to wait for the developers and the coders to make the platform so we can do something with it. We have a tech team, so sometimes things we want we don’t have to wait so long for our wishes as other users, but mostly our work in OpenSim has been about using it, breaking it, finding the issues, and reporting them back to the Core developers who can then fix what we have found which is broken.

In the 2 1/2 years we’ve been working on the platform, we’ve come from that initial sense of awe and wonderment at being in a new world running on our desktops; one that was unformed and simple: Ruth on a Dot. Ruth on a Dot wasn’t itself terribly exciting, but the sense of futurity, the sense of possibility, sweeping vision for what the platform could become was simply breathtaking. That was how it felt back in September 2007, when we brought up our first dot of land in our first OpenSim instance. Nothing worked quite right – there was no persistence, no physics, really not much to commend it, except for the fact you could create objects…and the vision of what it could become. That is what freedom looked like in September 2007: Ruth on a Dot. What it felt like was a path opening before us.

The path has led us to some amazing experiences. In 2008, in our research collaboration with IBM, we built Shengri La Spirit in a matter of days, on a platform that wasn’t really ready for what we wanted it do. But we persevered, the platform cooperated enough, and Spirit was built. We almost lost Shengri La Spirit – in fact, it was lost, for more than a year. But, and this is critical to understanding why a platform like OpenSim is so important and needed: after a year of that data lying in a heap, one of our team members was able to resurrect Spirit. It wasn’t lost, as so much of our work in Second Life has been lost.

Now Spirit is hosted on the ScienceSim grid as part of our research collaboration with Intel Labs. Anyone can log in, and walk through the trellis hall. Anyone can see the prim drift and shift that occurred as we fought, cajoled and encouraged something that wasn’t quite ready to do what we wanted it to do. There’s ample visual evidence of where the state of OpenSim was; we have never cleaned the build up since we feel it is a matter of historical record. We have the data in an OAR file. It is ours to do with as we please. But that is where we were, in 2008.

In 2009, we began a research collaboration with Intel Labs to focus on performance and content issues. We provided content, including large-scale, complicated regions, which the Intel engineers used as workloads and test cases. Our development of the 256,000 object region Shengri La Chamomile resulted in several performance patches contributed back to the OpenSim code base. Chamomile, too, serves as a benchmark: this is where the OpenSim codebase was in late 2009. It too, can be visited – however, the experience is challenging to anyone with less than robust hardware and bandwidth; and the viewers aren’t really up to rendering the scene. Chamomile stands as a challenge for viewer developers, and marks the state of development of OpenSim in 2009.

Now, in 2010, we’d like to showcase the first of what we expect to be many developments in 2010. For the past month, the team in ScienceSim has been head’s down building infrastructure designed to support our future work with OpenSim. We are developing content standards that govern how collections of content can be managed, curated and tracked. We have been developing tools that are useful for the audience of educators and enterprise using the ScienceSim grid. We have been defining policy and procedure (and documenting it as we go) to define best practices in settling new OpenSim-based grids for enterprise and education. We are creating, building, enhancing, and evolving all of this on top of the OpenSim code base, which is now robust enough to support the work we’re doing.

We are now standing on the threshold of something amazing, which has the same feeling of being an early OpenSim Ruth on a Dot in 2007.

The following images were snapped in ScienceSim grid yesterday, shortly after the regions had the terrain for the Yellowstone National Park loaded. The terrain data was provided by Dr. Brian Quinn, whose work with translating LiDAR data into Second Life and OpenSim terrain is well known by many. Brian provided the terrain at 1:10.38 scale, and it is loaded into a series of megaregions that stretch on as far as the viewer can see.

This is an included post about these regions from Mic Bowman, Intel’s lead engineer on this project, on Intel’s involvement with ScienceSim:

“As part of sizing the hw requirements for a mirror world project we’re exploring… we wanted to do some scalability tests on the capacity of individual simulators in terms of the total number of regions. we wanted baseline numbers that focus on just the most simple region configuration: a completely empty region with default terrain. That is… this is JUST simulator overhead.

All tests are done on one of our blades… dual proc, quad core with 16G ram running 64bit ubuntu. mono 2.6. and the tests are hosting 1024 regions in a 32×32 grid. The simulator configuration was our standard sciencesim config (XEngine, ODE, groups, wind, sun, etc).

The first configuration ran all 1024 regions in *one* simulator. I honestly figured this would crash quickly. It didn’t. We managed to get all 1024 regions created and running well enough to walk around.It did take almost 8 hours to start. The first couple regions were created in 1-2 seconds each. by the end, it was taking 4-5 minutes per region. Clearly there is something quadratic in there (stop using linear lists, they are evil! . But it could have been the mono garbage collector. who knows… not sure its worth too much investigation because I can’t imagine ever running a config like this for real. The simulator did crash when we opened the map in the viewer. The crash was caused because we ran out of sockets. While it was running, the simulator used just over 10G of ram and was running at about 700% CPU utilization (kind of scary to see load averages in the 900 range!).

The second configuration ran 16 simulators each with 64 regions. Startup took about 30 minutes (each of the 16 simulators avoided the quadratic “knee” we hit with the one big simulator). Consumed about 11G of memory and was again consuming essentially all cycles on the machine (completely idle regions aren’t very idle). All 16 simulators died just after startup with a “too many open files” error. Not sure what caused it, but all of them died loading the same terrain dll as part of some wildcard function looking for dlls. That one is an interesting bug.

The final configuration and the one we’re shooting for in the short term runs 16 simulators, each with an 8×8 megaregion. again startup was around 40 minutes. we might be able to cut that time down by starting all 16 simulators simultaneously rather than 4 at a time. I really just wanted to make sure that some of the spikes we see in startup didn’t cause failures (some race condition causes all threads to consume maximum processor cycles randomly on startup right now). and, well… it just worked. I figured the viewer would die horribly (it can’t handle 250K “real” prims very well) but it survived just fine. turn off far clip with 8×8 megaregions providing neighbors and you are “capable” of contacting simulators in a 24×24 region range!. the view is pretty cool though it seems to not go beyond 12×12. there are a bunch of problems (like you can’t see the terrain in the region you’re standing in)… but there are a LOT of little humps of terrain in view. Oh… and it takes a lot of patience to get everything loaded.

So… the conclusion… this opensim thing is pretty amazing! Good work everyone.

–mic”

End Quote.

OpenSim’s fourth year builds on the work of the past three years. We start 2010 with an OpenSim benchmark that shows how far it has come from 2007. What was accomplished last week has sweeping implications for enterprise application on a very large scale. It has implications for the use of OpenSim for modeling an array of different sorts of data. What was so breathtaking about standing in the Yellowstone terrain and looking around was the vision of how such a creation could be used: for geology education; for management of land, water, and mineral rights; showcasing real estate development; managing a national park in real time; modeling war games; the development of a created virtual space that could be used in common for all of these uses by an array of users. All of these use cases are possible. And they all start here, with a single sweeping expanse of virtual land that models a part of the real world, and which was instantiated in the space of less than a week.

→ 8 Comments

Posted in secondlife

ScienceSim Land Grant Program Overview & FAQ

Posted on January 18, 2010 by Shenlei | 1 Comment

We’ve had requests to repeat the presentation about the ScienceSim land grant program. Although most of the parcels have been assigned, there are a few left, so we’re providing this overview of the program for those who have been unable to attend our past presentations.

The ScienceSim land grant program is an opportunity for nonprofit entities to explore the use of the OpenSim platform for their educational, research, and scientific endeavors. The program ends on June 30, 2010. There is no cost for participation in the program. Management oversight is provided by the Fashion Research Institute, and grid administration is handled by Intel Labs. The ScienceSim grid is hosted by IEEE/ACM.

The ScienceSim grid originated as a part of an exhibit for the 3D Internet track for the Supercomputing 09 conference, held in November 2009. The multidisciplinary collaborators on the ScienceSim grid worked closely to develop a wide array of engaging exhibits, some of which were shown as part of Intel CTO Justin Rattner’s keynote speech at Supercomputing 09. As the conference approached, collaborators were concerned that the work they had accomplished in common, the research they had conducted, and the results they had achieved in common not be lost after the conference. To that end, it was agreed by these collaborators that they would work to create a formal organization which would serve as the catcher not only of their common work, including the code, content, and best practices they had created, but would also work to develop a stable distribution of OpenSim for the use of educators, researchers, and scientists. The distribution would include not just the code, but full documentation, best practices, and curated content.

In order to help raise awareness and attention about such an organization, the ScienceSim land grant program was developed and launched. Those of us who work extensively in OpenSim already know how flexible and extensible the platform is. It serves the needs of fashion design students as readily as it meets the requirements of scientists who want to show visualizations of large scale datasets and changes that occur to them over time. The low cost and ability of the platform to scale to accommodate the needs of multidisciplinary users has been amply proven to us, both through our own programs or through collaborating with other groups. Enabling other users to access this platform is, we feel, key to the evolution of the platform and its ecology of community, content, and user needs.

An active community of explorers and users will help us define the user needs for the platform, which in turn will help us define what a stable distribution should look like. In other words, we are all collaborating together to help move OpenSim forward.

Like Second Life regions, OpenSim regions are also 256mx256m square. We have divided our regions into 1/4 region parcels, with the edges around the parcels retained for use in ‘urban infrastructure’, eg, walkways and roads. Recipients of parcels in the ScienceSim land grant program includes a 100mx100m parcel, with each parcel containing 8,000 primitives. For those keeping score, yes, this means that the regions themselves can contain up to 35,000 primitives. While the servers running these regions are capable of much more (for example, Shengri La Chamomile currently has 256,000 primitive objects in the region) what we have found is that user experience degrades over 35,000 objects. We are therefore artificially limiting the regions to a mere 8,000 prims per recipient, in order to help manage user experience for all users.

In addition to the basic parcel, Fashion Research Institute is providing professional management oversight of the program, including managing the land covenant and other agreements. The program also includes access to premium, verified original content, licensed for use on the ScienceSim grid. This library of content includes 170 prefab buildings, developed in 6 styles, in 6 colors per style, in an array of sizes per style; 80 texture packs, each containing 12-20 unique textures; an array of avatar customization content for both male and female avatars (skins, shapes, eyes, hair, clothing, shoes, accessories); landscaping and hardscaping; sculpted animals, and more. The library also includes a collection of ‘basic’ scripting which has been written by the collaborators and donated to the scriptorium with the appropriate licenses. And, lastly, a proven orientation gateway system, courtesy of FRI, for users to orient their new users. This gateway has been used to orient more than 65,000 new users and can be used by land grant recipients for orientation of their new users.

What the program does not include is any additional region, content, or program development; any backend integration to existing programs; or consulting. Recipients must be able to handle their own development or they must have a budget to pay for any additional development they themselves cannot perform. The ScienceSim land grant program cannot provide these services.

Entities which have heavy tech integration needs or which require 24/7 customer service support should consider a commercial alternative such as ReactionGrid, SimHost, or 3D Hosting. The ScienceSim land grant is not a competitive alternative to a commercial hosting facility and should not be regarded as such.

The land grant program ends on June 30, 2010. At that time, the servers being used to host the regions will be re-purposed. They are not available to rent or hire and regions currently hosted on those servers must be migrated elsewhere. The data, research, content, and context the parcel recipients have developed in their region will not, however, be lost. Parcel recipients own their data. And they will be provided with their data in the form of an OAR (OpenSim Archive Resource) file at the end of the term. Depending on the size of the file, it will be emailed or placed on an FTP site where it can be picked up.

Either a full OAR file or a partial OAR file will be produced, depending on the future plans of the recipients. Participants who want to continue maintaining a presence on the ScienceSim grid after June 30, 2010, will need to make arrangements for their own hosting (either through internal resources or through commercial services). Monthly hosting costs vary widely depending on the technical facilities an entity has internally or their need to purchase external monthly hosting.

While OpenSim, and OpenSim-based grids are often compared to the ‘Wild West’ of virtual worlds, ScienceSim itself could not be further from such a statement. ScienceSim is hosted by the Institute of Electrical and Electronics Engineers (IEEE) and the Association of Computing Machinery (ACM). Weekly meetings are held during which representatives holding regions participate in the governance of the grid. Future directions of the grid are discussed and agreed upon at these meetings. Systems administration is provided by Intel Labs. There are weekly user meetings in which all users of the grid are welcome to participate. The ScienceSim land grant program itself is managed by FRI, and weekly governance meetings are held in the Homestead regions. All parcel recipients are welcome and encouraged to attend.

Naturally, along with governance comes rules. We too have rules with which we expect all recipients and their users to comply. We expect recipients to use their real identities combined with their real organizations. We expect our users to behave in accordance with the collaborative spirit of the community which has developed in ScienceSim, and to treat others with decency and respect. We expect the users of the grid to obey our land covenant, and to observe the terms of use of our content licenses. Some of our users encourage their children and other family members to explore OpenSim on our grid. We do not have any sort of content that encourages the ‘broader range of human expression’ beyond the bounds of expression that might be exhibited in a typical corporate or educational setting. We expect our users to conform to the same sort of expression.

This program is limited to representatives from entities and entities which are educational, research, or scientific organizations.

Commercial entities are not eligible for this program. If you already have a presence elsewhere on an OpenSim-based grid, you are not eligible. If you are an individual who just wants to explore, you are not eligible.

If you do not have basic ‘build’ abilities, and the ability to manage your program or project on your own, you won’t be a good fit for this program. Our collaborators are all busy with their own research, and while additional collaborations may well spring up as a result of proximity, you should not apply if you don’t have the basic skills you need to obtain one of our prefabs and get your parcel developed. You cannot assume there will be resources to help you develop your project or program.

The grid is called the ScienceSim grid, but that is not an implication about the sorts of programs or disciplines which will be engaging in these parcels over the next six months. Naturally, we have several Ed Tech programs which will be developing projects there, but we also have fashion design interns, legal interns, geology, game design, and even creative writing projects which will develop in these parcels. What do you want to explore in this six month window? You can use the space as teaching space, as lab space, as development space across an array of disciplines.

Sound like utopia? Almost, but of course, there’s always some fine print. We like to think we’ve kept ours to a minimum, but we still have a little. Fashion Research Institute holds a formal research agreement with Intel Labs. We have completed the due diligence process and have signed the research contract that governs our engagement on the ScienceSim grid. We have agreed to serve as the administrators of the ScienceSim land grant program. What this means is that any parcel recipient must sign a legally binding agreement with FRI before a parcel can be awarded. We have worked with our attorneys to create a short, 2-page agreement that is written in as close to plain English as we could get it. The terms of the agreement are not negotiable.

Our lawyers tell us we cannot sign agreements with avatars. Real identities, real organizations only, and you should assume basic due diligence will be conducted. Requests from non-organizational e-mail addresses will be met with a standard form response to send us a request out of your official e-mail account. We cannot engage with individuals who aren’t affiliated with the organization they purport to represent.

Want to apply for a parcel? It’s easy. Send us your real identity, your real affiliation, and a brief 2-3 sentence (even a paragraph, we read fast) description out of your affiliation e-mail. We review applications and turn them around within 24 hours. If you do not hear from us within 24 hours, it means your request was caught in a spam filter. Please resend in such cases.

The application e-mail address is: admin @ fashionresearchinstitute.com