Getting project to work with 32-bit OS

Jul 12, 2010 at 2:54 PM

Hey,

 

I thought I would add a note here since we were just discussing this topic on stackoverflow here (http://stackoverflow.com/questions/3223060/working-with-large-collections-in-db40-net).  What exactly would be entailed in getting the project to work on a 32-bit system.  I tried downloading the source and replacing your modified Winterdom.FileMap.IO with the original, as taken from GitHub.  However, I ran into problems on a line of ViewManager.cs, inside of AddViewToThreadPool:

            _viewThreadPool[threadId] = mvs = _map.MapAsStream();

Essentially, the modified Winterdom.FileMap.IO defines a method called MapAsStream that is not part of the standard distribution.  Since they do not seem to have anywhere near the same signature.  Any help here would be appreciated!

I would really love to try this within my own project but to do so I need to be able to use this library on 32-bit systems.

p.s. The NTFS.Sparse project does not appear to be in the source control tree, but rather somewhere else on your hard drive.  Would you add it please?  The project will not compile otherwise.

Coordinator
Jul 13, 2010 at 8:19 AM

Thanks for posting..

I'll try and get some time later today to look at this. Basically the code has to be changed to allow the view to be moved around the file, instead of creating a view over the whole file. And probably add a setting for how large the view is allowed to grow, and the maximum number of views to hold at one time.

And I'll remove the NTFS.Sparse, as it's not in use yet :) But it will be in later versions to optimize allocation of disk space.

Coordinator
Jul 13, 2010 at 7:09 PM

Check out the Experimental32bit branch in the source tree. It's a back-port with dynamic view. It passed all the tests, but seems to have some file cleanup problems under threading.

You can adjust the map view sizes in the code.

The reason you couldn't find the MapAsStream method was because it was added with another patch, which took me a while to google up :) 

Jul 14, 2010 at 12:54 AM
Awesome, will check it out now. Thanks!
Jul 14, 2010 at 1:05 AM
Just tried to plug it in but now I'm getting this error when I try to declare a Dictionary<long,IUnit> where IUnit is a domain object of mine. Anything I can do to force it to specify a serializer to use? I'd even use protobuf if possible! Super-speedy. Unhandled Exception: System.TypeInitializationException: The type initializer for 'mAdcOW.DataStructures.DictionaryBacking.BackingUnknownSize`2' threw an exception. ---> mAdcOW.Serializer.SerializerException: No serializer available for the type at mAdcOW.Serializer.Factory`1.PickOptimalSerializer() in C:\dev\mmf-experimental32bit\mAdcOW.Serializer\Factory.cs:line 43 at mAdcOW.Serializer.Factory`1.GetSerializer() in C:\dev\mmf-experimental32bit\mAdcOW.Serializer\Factory.cs:line 21 at mAdcOW.DataStructures.DictionaryBacking.BackingUnknownSize`2..cctor() in C:\dev\mmf-experimental32bit\mAdcOW.DataStructures\DictionaryBacking\BackingUnknownSize.cs:line 25 --- End of inner exception stack trace --- at mAdcOW.DataStructures.DictionaryBacking.BackingUnknownSize`2.SetDefaultKeyValueSize() in C:\dev\mmf-experimental32bit\mAdcOW.DataStructures\DictionaryBacking\BackingUnknownSize.cs:line 149 at mAdcOW.DataStructures.DictionaryBacking.BackingUnknownSize`2..ctor(Stringpath, Int32 capacity, Boolean keepFile, String dictionaryName) in C:\dev\mmf-experimental32bit\mAdcOW.DataStructures\DictionaryBacking\BackingUnknownSize.cs:line 74 at mAdcOW.DataStructures.Dictionary`2..ctor(String path, Int32 capacity) in C:\dev\mmf-experimental32bit\mAdcOW.DataStructures\Dictionary.cs:line 40 at Db4oTest.TestEngineSession.Start() in C:\dev\canceis-madcow-test\STC.EI.Canceis\StressTest\TestEngineSession.cs:line 49 at StressTest.Program.Main(String[] args) in C:\dev\canceis-madcow-test\STC.EI.Canceis\StressTest\Program.cs:line 47 BTW, thanks for doing this. much appreciated.
Coordinator
Jul 14, 2010 at 11:14 AM

You can try to update the protobuf-net code to the latest version, or decorate your domain object with [DataContract] [DataMember] attributes. This should make it serializable.

Also remember that your object needs an empty constructor in order to instantiate the class. Special handling is done for byte[] and strings, as they don't have an empty constructor, but are used frequently.

An improvement for the serializers would be to manage instantiation with any type of constructor, not just an empty one.

Jul 14, 2010 at 12:03 PM
I will give that a try.  I'll look into the protobuf code as well since it appears to be quite fast!

Thanks again.

On Wed, Jul 14, 2010 at 7:14 AM, Wobba <notifications@codeplex.com> wrote:

From: Wobba

You can try to update the protobuf-net code to the latest version, or decorate your domain object with [DataContract] [DataMember] attributes. This should make it serializable.

Also remember that your object needs an empty constructor in order to instantiate the class. Special handling is done for byte[] and strings, as they don't have an empty constructor, but are used frequently.

An improvement for the serializers would be to manage instantiation with any type of constructor, not just an empty one.

Read the full discussion online.

To add a post to this discussion, reply to this email (mmf@discussions.codeplex.com)

To start a new discussion for this project, email mmf@discussions.codeplex.com

You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe on CodePlex.com.

Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at CodePlex.com


Coordinator
Jul 15, 2010 at 11:41 AM

The protobuf serializer is very fast indeed. At least for classes. For structs and value types my current code is probably as fast as you get, as it does direct memory copy.

That's why I added it. I just haven't had the time to update the code I currently use.

Good luck, and I hope the project will work for your scenario.