XEngine Forums

[Asgard]

As of today, November 5, 2003, the move from XEngine's previous smart pointer RefPtr to boost::shared_ptr is complete and committed to CVS. A lot of other changes have happened since the last time I checked something into CVS. The following is a detailed change log and some other notes. Please take the time to read it all and do not update from CVS until you understand all the implications outlined below.

Change Log

General
- Switched over to standard naming convention for macros, e.g. XEngine_Precompiled_Headers now becomes XENGINE_PRECOMPILED_HEADERS
- Switched from own RefPtr smart pointer to boost::shared_ptr (see below for details).
- Added build files for wxGTK2.
- Numerous small bug fixes and documentation cleanups. Made the code more exception-safe in various areas.

XEngineMath
- A number of Vector and Matrix class constructors have been made explicit. This might require some explicit conversions to be added in client code, for example when trying to use a Vector3 where a Vector4 is expected as parameter.
- Added Matrix4x4::InvertRotationTranslationMatrix and similar routines for faster inversion of translation/rotation-only matrices.

XEngineCore
- The classes RefPtr and RefCountedObject have been removed.
- RenderResource-derived classes are now reference counted with boost::shared_ptr instead of RefPtr.
- RenderTarget-derived classes are now also reference counted (with boost::shared_ptr).
- Render windows must have an instance of RenderWindowParent as parent window. This is transparently handled for clients by RenderContext::CreateRenderTarget and does not require any client source code changes in most cases. Please see the RenderWindow and RenderWindowParent class documentation for details on render window lifetime management issues.
- Added a new template helper class named RenderBufferLocker. It should be used instead of calling RenderBuffer::Lock and RenderBuffer::Unlock directly.
- RSZBuffer was renamed to RSDepthBuffer since a depth buffer need not necessarily be a buffer that stores z values.
- Completely reimplemented the Cg bindings parser used by XEngine's internal Cg runtime with Spirit, and fixed some bugs related to it with the D3D 9 renderer.
- Added better handling for array parameters to the internal Cg runtime. Whereas until now it only used to be possible to set the entire array at once it is now possible to set certain elements of arrays or sub-arrays of multi-dimensional arrays by specifying the element to be set in square brackets.
For example, given an array parameter, uniform float4x4 matrices[2][2], the parameter names matrices, matrices[0], matrices[0][1], and similar can be used in calls to ShaderProgram::SetParameter. This does not yet work for vertex attributes in practice, simply because vertex attribute arrays are not yet supported by the Cg compiler, although XEngine's internal Cg runtime could handle it already.
- Added parameter shadowing for sampler parameters, see ShaderProgram::SetSamplerParameter.
- Added a high-precision Timer class, which is now used in all the samples.

XEngineRenderers
- Changed some Cg shader classes to accommodate for the new internal Cg runtime.
- Various fixes here and there, most importantly in the ATI_fragment_shader classes.

XEngineUtil
- Removed the old Mesh and PrimitiveGroup classes and replaced them with a new BufferMesh class.
- Removed the MeshLoader hierarchy of classes. They are replaced by new model loader classes that also support animation.
- Added a ModelMD2 class that represents animated MD2 models that are loaded by the ModelLoaderMD2 class. This class will probably get a bit of an overhaul in the near future.
- The 3ds loader has been removed for the time being until it is rewritten to use the new BufferMesh class and also support animation in the near future.
- Added an FFT-based waves tile class.
- Added a Gestner model-based waves tile class.
- Added a skybox class.
- Expect XEngineUtil to be rather instable for the next few months as I try out different things. In particular, this might not be the last model loader or mesh class design.

XEngine and shared_ptr

The biggest change with this CVS commit is the move away from XEngine's own smart pointer class, RefPtr, to boost::shared_ptr.
Since this is a big interface change, I've collected some notes on correct boost::shared_ptr usage here. shared_ptr is a non-intrusive reference pointer, so it doesn't require the reference counted objects to carry the reference count themselves (as was the case with RefPtr).
Instead, the reference count is stored separately by the shared_ptr. In the debug build of XEngine this shared reference count is allocated on the heap with the new operator. In release build BOOST_SP_USE_QUICK_ALLOCATOR is defined (in XEngineConfig.h), which will make shared_ptr use a small object allocator for the shared reference counts. This quick allocator allocates chunks of memory at program startup that are not deallocated until program termination by the OS. Note that this is not a memory leak even though some memory diagnosis tools will report it as such.

The basic rules for shared_ptr usage in XEngine are this:
- RenderResource and RenderTarget-derived classes are always passed around as shared_ptrs. So where you'd have e.g. RefPtr<VertexBuffer> before you should now have boost::shared_ptr<VertexBuffer>.
- When the RenderContext that created a RenderResource or RenderTarget gets destroyed (by calling the delete operator on it), _all_ shared_ptr instances holding pointers to RenderResource or RenderTarget objects must have already been destroyed (this is the same behavior as before with RefPtr). I was looking for ways to also use shared_ptr to reference count RenderContext objects, so that the RenderContext would be kept alive as long as there are RenderResources or RenderTargets created by it still in existence, but couldn't find a way to avoid cyclic references. If someone has any good design ideas (apart from using a garbage collecting smart pointer implementation), I'm all open to suggestions.
- Use const boost::shared_ptr<T>& when passing shared_ptrs to functions, just as with any other class types.
- Use boost::static_pointer_cast<T> and boost::dynamic_pointer_cast to cast shared_ptrs in cases where you want to store the casted shared pointer somewhere for later use. Do not do something like static_cast<T*>(pSharedPtrObj.get()) and then assign the result to a new shared_ptr. This will result in double deletion of the pointed to object.
- Avoid cyclic references of shared_ptr objects. That is, a shared reference counted object A must not contain a shared pointer to an shared counted object B which in turn has a shared pointer to A. This is the same requirement as with RefPtr before. boost::weak_ptr can be used to break shared_pt reference cycles.

NVIDIA Drivers, Render Textures, and VBO

With the latest NVIDIA Windows OpenGL drivers (5x series and also 45.xx) there seems to be an issue when using ARB_vertex_buffer_object buffers in multiple rendering contexts that share objects. This manifests itself in access violations in calls to glDrawArrays/glDrawElements in nvoglnt.dll (the OpenGL part of the NVIDIA driver). This seems to be a driver bug because the same code runs flawlessly on ATI drivers and with NVIDIA's Linux drivers. However, I haven't had time to investigate any further or report the issue to NVIDIA. Until I do and if you need render textures with the OpenGL renderer, you can simply turn off VBO support. In the file /src/Renderers/RendererOGL13/XRenderBufferBaseOGL13.cpp right at the beginning where it says

[sharpie-guest]

Just tried building the VS.NET 2003 solution, here's the first compilation error:

Compiling...
XEngineCore.cpp
C:\Documents and Settings\bellachunk\Desktop\xengine\biuldroot\Boost\boost\spirit\core\primitives\impl\primitives.ipp(174) : warning C4244: 'return' : conversion from 'int' to 'char', possible loss of data
C:\Documents and Settings\bellachunk\Desktop\xengine\biuldroot\Boost\boost\spirit\core\primitives\impl\primitives.ipp(177) : warning C4244: 'return' : conversion from 'int' to 'char', possible loss of data
..\..\include\Core\ShaderCompilers\XParser.h(67) : error C2661: 'boost::spirit::position_iterator<IteratorT>::__ctor' : no overloaded function takes 2 arguments
with
[
IteratorT=const wxChar *
]
and
[
IteratorT=const wxChar *
]
..\..\include\Core\ShaderCompilers\XParser.h(61) : while compiling class-template member function 'const XEngine::Core::ShaderCompilers::Grammar_CgBindings::ParserResults &XEngine::Core::ShaderCompilers::Parser<GrammarT,SkipGrammarT>::Parse(const wxString &)'
with
[
GrammarT=XEngine::Core::ShaderCompilers::Grammar_CgBindings,
SkipGrammarT=XEngine::Core::ShaderCompilers::SkipGrammar_CgBindings
]
..\..\include\Core\ShaderCompilers\XParser_CgBindings.h(130) : see reference to class template instantiation 'XEngine::Core::ShaderCompilers::Parser<GrammarT,SkipGrammarT>' being compiled
with
[
GrammarT=XEngine::Core::ShaderCompilers::Grammar_CgBindings,
SkipGrammarT=XEngine::Core::ShaderCompilers::SkipGrammar_CgBindings
]

[Asgard]

What version of Spirit are you using? You should use 1.7.0 which you can grab from http://spirit.sourceforge.net. Simply extract the distribution into your current Boost directory overwriting the old Spirit 1.6.1 which comes with Boost 1.30.2.

[sharpie]

That must be it, I didn't notice that anywhere in the docs but I didn't look to carefully before trying to build.

Great job on this release, it really seems you've put xengine in line for a lot of painless feature improvements over the next releases.

If you start to work on the meshloader for 3DSM please let me know. I would like to implement one for Softimage XSI at the same time.

Btw, I meant to let you know I found another great scripting language/engine even more goaled towards games development, its called Squirrell and it uses a combo of GC and ref counting to completely elliminate CPU bursts for deterministic finalization. Looks about as easy/difficult to build bindings for and its syntax is C oriented, vs. Io's whacky new style.

I'll be converting my sample over to Squirrell first chance I get and I'll let you know how it goes, if you're still interested in the scripting side of things.

[Asgard]

[sharpie-guest]

[sharpie-guest]

Now I can't find the Squirrel site. Do you have it saved Asgard? I searched on GOogle and Flipcode and I keep getting this BeOS squirrell. That can't be what i was looking at before.

[Asgard]

It's http://squirrel.sourceforge.net

[Won]

Wow. This stuff is pretty cool. I a, thinking about doing something similar to this, but now I might just use XEngine.

May be kind of late, but why use boost::shared_ptr instead of ::intrusive_ptr? The intrusive_ptr is smaller and faster, particularly when you consider memory access coherency and multithreading. Maybe performance is not an issue where you use it, but if you're going through the trouble of adding a small-object allocator...

-Won

[Asgard]