suitable for multiple processes?

Nov 13, 2009 at 12:40 PM

Thanks for this wonderful contribution.

I am not familiar with Memory Mapped Files, but as far I understand they can be used by different processes.

So I wonder if mAdcOW is designed to be used in a multi-process scenario?
I think of multiple terminalserver-sessions accessing the same structure (central cache),
possible?

Regards, Christoph

Coordinator
Nov 14, 2009 at 3:14 PM
christoph_ch68 wrote:

Thanks for this wonderful contribution.

I am not familiar with Memory Mapped Files, but as far I understand they can be used by different processes.

So I wonder if mAdcOW is designed to be used in a multi-process scenario?
I think of multiple terminalserver-sessions accessing the same structure (central cache),
possible?

Regards, Christoph

It's certainly possible for memory mapped files to be shared by several processes. And there are examples of this where one process writes, and others read.

The current implementation in "Disk based data structures" is designed as a replacement for the current List/Dictionary types, and creates a new file upon creation, and disposes it when done. Extending it to provide a backing file upon creation is a natural extension. By having a separate file or padding in a header which stores information about the datatypes/count and other states, the backing file could be shared by several processes.

I'll add it to the feature list.

Nov 18, 2009 at 9:11 AM

this is good news, thanks for putting it to the feature list.

As soon the project has moved to .net 4.0 i will have a closer look,
hopefully the multi process sample will be available then.

anyway, great stuf

Jul 16, 2012 at 8:39 PM
wobba wrote:
christoph_ch68 wrote:

Thanks for this wonderful contribution.

I am not familiar with Memory Mapped Files, but as far I understand they can be used by different processes.

So I wonder if mAdcOW is designed to be used in a multi-process scenario?
I think of multiple terminalserver-sessions accessing the same structure (central cache),
possible?

Regards, Christoph

It's certainly possible for memory mapped files to be shared by several processes. And there are examples of this where one process writes, and others read.

The current implementation in "Disk based data structures" is designed as a replacement for the current List/Dictionary types, and creates a new file upon creation, and disposes it when done. Extending it to provide a backing file upon creation is a natural extension. By having a separate file or padding in a header which stores information about the datatypes/count and other states, the backing file could be shared by several processes.

I'll add it to the feature list.


Has the concurent read functionality been incorporated into the latest .NET 4 build? If it has, it would appear that this implementation could be a viable replacement for the .NET ConcurrentDictionary class. It that assessment correct?

Thanks...

Coordinator
Jul 17, 2012 at 6:57 PM

Hi,

The implementation is thread safe and has been thread safe from the first release, and can be used instead of the concurrent dictionary.

So they are actually providing more than a replacement for the regular dict in 3.5. What's missing for .NET4is to use the mmf library which is built-in, instead of the current implementation. You can also provide a file as input, thus using it as a simple database.

There is no timeline for the .NET4 release, but hopefully by the end of August we should have a stable release.

Thanks,
Mikael 

Jul 17, 2012 at 7:56 PM

I am aware of some of the differences between the two dictionaries. I have some good benchmark data comparing the two. 

I was referring to concurrent access (simultaneous reads). As I understand it, the 3.5 Dictionary locks on reads/writes, while the concurrent dictionary allows both simultaneously. In your implementation, is the read still locking for each thread, meaning the others wait until the current lock is released?

Thanks, Warren

Coordinator
Jul 17, 2012 at 8:08 PM

Hi,

The code uses ReaderWriterLockSlim in all places to allow multiple simultaneous reads, as reads are usually more frequent than writes. I very seldom use blocking locks in my code :)

-mikael

Jul 17, 2012 at 8:26 PM

That's great news about simutaneous reads, as that, coupled with solid state disks, is a real performance enabler for my system.  You made my day! :-) Have a good vacation...

Warren