[music-dsp] Hosting playback module for samples

[Mark Garvin]

I realize that this is slightly off the beaten path for this group,
but it's a problem that I've been trying to solve for a few years:

I had written software for notation-based composition and playback of
orchestral scores. That was done via MIDI. I was working on porting
the original C++ to C#, and everything went well...except for playback.
The world has changed from MIDI-based rack-mount samplers to computer-
based samples played back via hosted VSTi's.

And unfortunately, hosting a VSTi is another world of involved software
development, even with unmanaged C++ code. Hosting with managed code
(C#) should be possible, but I don't think it has been done yet. So
I'm stuck. I've spoken to Marc Jacobi, who has a managed wrapper for
VST C++ code, but VSTi hosting is still not that simple. Marc is very
helpful and generous, and I pester him once a year, but it remains an
elusive problem.

It occurred to me that one of the resourceful people here may have
ideas for working around this. What I'm looking for, short term, is
simply a way to play back orchestral samples or even guitar/bass/drums
as a way of testing my ported C# code. Ideally send note-on, velocity,
note-off, similar to primitive MIDI. Continuous controller for volume
would be icing.

Any ideas, however abstract, would be greatly appreciated.

MG
NYC

[Douglas Repetto]

PortAudio!
http://www.portaudio.com


best,
douglas

Post by Mark Garvin
I realize that this is slightly off the beaten path for this group,
I had written software for notation-based composition and playback of
orchestral scores. That was done via MIDI. I was working on porting
the original C++ to C#, and everything went well...except for playback.
The world has changed from MIDI-based rack-mount samplers to computer-
based samples played back via hosted VSTi's.
And unfortunately, hosting a VSTi is another world of involved software
development, even with unmanaged C++ code. Hosting with managed code
(C#) should be possible, but I don't think it has been done yet. So
I'm stuck. I've spoken to Marc Jacobi, who has a managed wrapper for
VST C++ code, but VSTi hosting is still not that simple. Marc is very
helpful and generous, and I pester him once a year, but it remains an
elusive problem.
It occurred to me that one of the resourceful people here may have
ideas for working around this. What I'm looking for, short term, is
simply a way to play back orchestral samples or even guitar/bass/drums
as a way of testing my ported C# code. Ideally send note-on, velocity,
note-off, similar to primitive MIDI. Continuous controller for volume
would be icing.
Any ideas, however abstract, would be greatly appreciated.
MG
NYC
--
subscription info, FAQ, source code archive, list archive, book reviews,
dsp links
http://music.columbia.edu/cmc/music-dsp
http://music.columbia.edu/mailman/listinfo/music-dsp

[Thomas Rehaag]

Hi Mark,

if you just need a simple VST2 Windows host there's enough source code
that will give you access tho VSTi. You could use the MiniHost from the
VST2 SDK (hope you've got a copy. The download/support is gone since
some weeks). And of course you can have a look at Hermann Seib's VstHost:
http://www.hermannseib.com/programs/vsthostsrc.zip
but the MiniHost will be less complicated and sufficient.

I've just wrote a host for a job and it was the same situation: C# on
the other side. So we decided to use "shared memory" (CreateFileMapping
...) for communication. Works fine.

Cheers,

Thomas

Post by Mark Garvin
I realize that this is slightly off the beaten path for this group,
I had written software for notation-based composition and playback of
orchestral scores. That was done via MIDI. I was working on porting
the original C++ to C#, and everything went well...except for playback.
The world has changed from MIDI-based rack-mount samplers to computer-
based samples played back via hosted VSTi's.
And unfortunately, hosting a VSTi is another world of involved software
development, even with unmanaged C++ code. Hosting with managed code
(C#) should be possible, but I don't think it has been done yet. So
I'm stuck. I've spoken to Marc Jacobi, who has a managed wrapper for
VST C++ code, but VSTi hosting is still not that simple. Marc is very
helpful and generous, and I pester him once a year, but it remains an
elusive problem.
It occurred to me that one of the resourceful people here may have
ideas for working around this. What I'm looking for, short term, is
simply a way to play back orchestral samples or even guitar/bass/drums
as a way of testing my ported C# code. Ideally send note-on, velocity,
note-off, similar to primitive MIDI. Continuous controller for volume
would be icing.
Any ideas, however abstract, would be greatly appreciated.
MG
NYC
--
subscription info, FAQ, source code archive, list archive, book reviews, dsp links
http://music.columbia.edu/cmc/music-dsp
http://music.columbia.edu/mailman/listinfo/music-dsp

[Ross Bencina]

Hi Mark,

I'm not really sure that I understand the problem. Can you be more
specific about the problems that you're facing?

Personally I would avoid managed code for anything real-time (ducks).

You're need to build a simple audio engine (consider PortAudio or the
ASIO SDK). And write some VSTi hosting code using the VST SDK. It's this
last bit that will require some work. But if you limit yourself to a
small number of supported plugins to begin with it should not be too
hard. MIDI scheduling in a VSTi is not particularly challenging -- the
plugins do the sub-buffer scheduling, you just need to put together a
frame of MIDI events for each audio frame.

If there's any kind of "synchronisation with the outside world" things
will get trickier, but if you can clock the MIDI time off the
accumulated sample position it's not hard.

Other details:

You're going to need some kind of lock-free communication mechanism with
your audio callback (e.g. some kind of FIFO). I guess the main
approaches would be to either (A) schedule MIDI events ahead of time
from your C# code and use a priority queue (Knuth Heap is easy and
relatively safe for real-time) in the audio thread to work out when to
schedule them; or (B) maintain the whole MIDI sequence in a vector and
just play through it from the audio thread. Then you need a mechanism to
update the sequence when it changes (just swap in a new one?).

Cheers,

Ross

Post by Mark Garvin
I realize that this is slightly off the beaten path for this group,
I had written software for notation-based composition and playback of
orchestral scores. That was done via MIDI. I was working on porting
the original C++ to C#, and everything went well...except for playback.
The world has changed from MIDI-based rack-mount samplers to computer-
based samples played back via hosted VSTi's.
And unfortunately, hosting a VSTi is another world of involved software
development, even with unmanaged C++ code. Hosting with managed code
(C#) should be possible, but I don't think it has been done yet. So
I'm stuck. I've spoken to Marc Jacobi, who has a managed wrapper for
VST C++ code, but VSTi hosting is still not that simple. Marc is very
helpful and generous, and I pester him once a year, but it remains an
elusive problem.
It occurred to me that one of the resourceful people here may have
ideas for working around this. What I'm looking for, short term, is
simply a way to play back orchestral samples or even guitar/bass/drums
as a way of testing my ported C# code. Ideally send note-on, velocity,
note-off, similar to primitive MIDI. Continuous controller for volume
would be icing.
Any ideas, however abstract, would be greatly appreciated.
MG
NYC
--
subscription info, FAQ, source code archive, list archive, book reviews, dsp links
http://music.columbia.edu/cmc/music-dsp
http://music.columbia.edu/mailman/listinfo/music-dsp

[Mark Garvin]

1. Re: Hosting playback module for samples (Ross Bencina)
Hi Mark,
I'm not really sure that I understand the problem. Can you be more
specific about the problems that you're facing?

Hi Ross,

Specific: (Forgive me if you know all of this): My code is C#, but
there is no simple way to bridge the gap between the managed realm
and a module that can play orchestral samples.

Most sample banks these days seem to be in NKI format (Native
Instruments). They have the ability to map ranges of a keyboard into
different samples so the timbres don't become munchkin-ized or
Vader-ized. IOW, natural sound within each register.

A playback engine is typically something like Native Instruments'
Kontakt, which is 'hosted' by the main program (my composition
software, for ex). then NI Kontakt can load up NKI files and
deliver sound when it receives events.

The whole process of linking events, etc is what usually what
stymies programmers who are new to VST-based programming. And
even many who are familiar.

Personally I would avoid managed code for anything real-time (ducks).

Actually, C# can be faster than pre-compiled code! The first time a given
section of code runs, the JIT (just in time) compiler replaces the 'IL'
code with highly optimized machine code. Since the JIT module can
test the CPU that it running on, it can optimize for that particular
CPU and chip set. Early on, one of the popular book authors (Richard
Grimes) coded FFT algorithms in both C++ and C#, and C# was faster.

You're need to build a simple audio engine (consider PortAudio or the
ASIO SDK). And write some VSTi hosting code using the VST SDK. It's this
last bit that will require some work. But if you limit yourself to a
small number of supported plugins to begin with it should not be too
hard. MIDI scheduling in a VSTi is not particularly challenging -- the
plugins do the sub-buffer scheduling, you just need to put together a
frame of MIDI events for each audio frame.

That's inspiring. I'm not sure that this is done in the same way as a
regular plugin though. And I believe it's pretty difficult to host a
VSIi in managed code. That is pretty much the crux of the problem right
there. I've heard of a lot of people who started the project but were
never aboe to get it off te ground.

If there's any kind of "synchronisation with the outside world" things
will get trickier, but if you can clock the MIDI time off the
accumulated sample position it's not hard.

I could do without sync to external for now.

... I guess the main
approaches would be to either (A) schedule MIDI events ahead of time
from your C# code and use a priority queue (Knuth Heap is easy and
relatively safe for real-time) in the audio thread to work out when to
schedule them; or (B) maintain the whole MIDI sequence in a vector and
just play through it from the audio thread. Then you need a mechanism to
update the sequence when it changes (just swap in a new one?).

The internals of a VSTi host are beyond me at present. I was hoping
for some simple thing that could be accessed by sending MIDI-like events
to a single queue.

Thanks to both who responded. I've signed up for the PortAudio mailing
list. Maybe they have something.

Mark G.
NYC

[Ross Bencina]

Hello Mark,

Post by Mark Garvin
Most sample banks these days seem to be in NKI format (Native
Instruments). They have the ability to map ranges of a keyboard into
different samples so the timbres don't become munchkin-ized or
Vader-ized. IOW, natural sound within each register.
A playback engine is typically something like Native Instruments'
Kontakt, which is 'hosted' by the main program (my composition
software, for ex). then NI Kontakt can load up NKI files and
deliver sound when it receives events.
The whole process of linking events, etc is what usually what
stymies programmers who are new to VST-based programming. And
even many who are familiar.

Yes the VST SDK is not the best documented in the world.

Post by Mark Garvin

Post by Ross Bencina
Personally I would avoid managed code for anything real-time (ducks).

Actually, C# can be faster than pre-compiled code!

Speed has nothing to do with real-timeness.

Real-time is all about deterministic timing. Runtime-JIT and garbage
collection both mess with timing. It may be that CLR always JITs at load
time. That doesn't save you from GC (of course there are ways to avoid
GC stalls in C#, but if you just used a deterministic language this
wouldn't be necessary).

Post by Mark Garvin

Post by Ross Bencina
You're need to build a simple audio engine (consider PortAudio or the
ASIO SDK). And write some VSTi hosting code using the VST SDK. It's this
last bit that will require some work. But if you limit yourself to a
small number of supported plugins to begin with it should not be too
hard. MIDI scheduling in a VSTi is not particularly challenging -- the
plugins do the sub-buffer scheduling, you just need to put together a
frame of MIDI events for each audio frame.

That's inspiring. I'm not sure that this is done in the same way as a
regular plugin though.

I'm not sure what you mean by "a regular plugin".

I have a commercial VST host on the market so I do know what I'm talking
about.

Post by Mark Garvin
And I believe it's pretty difficult to host a
VSIi in managed code. That is pretty much the crux of the problem right
there. I've heard of a lot of people who started the project but were
never aboe to get it off te ground.

So you're insisting on using C# for real-time audio? As noted above I
think this is a bad idea. There is no rational reason to use C# in this
situation.

Just use unmanaged C++ for this part of your program. Things will go
much better for you. Not the least because both real-time audio APIs and
the VST SDK are unmanaged components.

Post by Mark Garvin

Post by Ross Bencina
If there's any kind of "synchronisation with the outside world" things
will get trickier, but if you can clock the MIDI time off the
accumulated sample position it's not hard.

I could do without sync to external for now.

Post by Ross Bencina
... I guess the main
approaches would be to either (A) schedule MIDI events ahead of time
from your C# code and use a priority queue (Knuth Heap is easy and
relatively safe for real-time) in the audio thread to work out when to
schedule them; or (B) maintain the whole MIDI sequence in a vector and
just play through it from the audio thread. Then you need a mechanism to
update the sequence when it changes (just swap in a new one?).

The internals of a VSTi host are beyond me at present. I was hoping
for some simple thing that could be accessed by sending MIDI-like events
to a single queue.

I'm sure there are people who will licence you something but I don't
know of an open source solution. JUCE might have something maybe?

Ross.

[Danny]

If i understand correctly, Juce would be the solution.
You say you already have the working c++ code, so you could use that and add an audioprocessor from juce to do your playback.

Post by Ross Bencina
Hello Mark,

Post by Mark Garvin
Most sample banks these days seem to be in NKI format (Native
Instruments). They have the ability to map ranges of a keyboard into
different samples so the timbres don't become munchkin-ized or
Vader-ized. IOW, natural sound within each register.
A playback engine is typically something like Native Instruments'
Kontakt, which is 'hosted' by the main program (my composition
software, for ex). then NI Kontakt can load up NKI files and
deliver sound when it receives events.
The whole process of linking events, etc is what usually what
stymies programmers who are new to VST-based programming. And
even many who are familiar.

Yes the VST SDK is not the best documented in the world.

Post by Mark Garvin

Post by Ross Bencina
Personally I would avoid managed code for anything real-time (ducks).

Actually, C# can be faster than pre-compiled code!

Speed has nothing to do with real-timeness.
Real-time is all about deterministic timing. Runtime-JIT and garbage collection both mess with timing. It may be that CLR always JITs at load time. That doesn't save you from GC (of course there are ways to avoid GC stalls in C#, but if you just used a deterministic language this wouldn't be necessary).

Post by Mark Garvin

Post by Ross Bencina
You're need to build a simple audio engine (consider PortAudio or the
ASIO SDK). And write some VSTi hosting code using the VST SDK. It's this
last bit that will require some work. But if you limit yourself to a
small number of supported plugins to begin with it should not be too
hard. MIDI scheduling in a VSTi is not particularly challenging -- the
plugins do the sub-buffer scheduling, you just need to put together a
frame of MIDI events for each audio frame.

That's inspiring. I'm not sure that this is done in the same way as a
regular plugin though.

I'm not sure what you mean by "a regular plugin".
I have a commercial VST host on the market so I do know what I'm talking about.

Post by Mark Garvin
And I believe it's pretty difficult to host a
VSIi in managed code. That is pretty much the crux of the problem right
there. I've heard of a lot of people who started the project but were
never aboe to get it off te ground.

So you're insisting on using C# for real-time audio? As noted above I think this is a bad idea. There is no rational reason to use C# in this situation.
Just use unmanaged C++ for this part of your program. Things will go much better for you. Not the least because both real-time audio APIs and the VST SDK are unmanaged components.

Post by Mark Garvin

Post by Ross Bencina
If there's any kind of "synchronisation with the outside world" things
will get trickier, but if you can clock the MIDI time off the
accumulated sample position it's not hard.

I could do without sync to external for now.

Post by Ross Bencina
... I guess the main
approaches would be to either (A) schedule MIDI events ahead of time
from your C# code and use a priority queue (Knuth Heap is easy and
relatively safe for real-time) in the audio thread to work out when to
schedule them; or (B) maintain the whole MIDI sequence in a vector and
just play through it from the audio thread. Then you need a mechanism to
update the sequence when it changes (just swap in a new one?).

The internals of a VSTi host are beyond me at present. I was hoping
for some simple thing that could be accessed by sending MIDI-like events
to a single queue.

I'm sure there are people who will licence you something but I don't know of an open source solution. JUCE might have something maybe?
Ross.
--
subscription info, FAQ, source code archive, list archive, book reviews, dsp links
http://music.columbia.edu/cmc/music-dsp
http://music.columbia.edu/mailman/listinfo/music-dsp

[Michael Gogins]

For straight sample playback, the C library FluidSynth, you can use it via
PInvoke. FluidSynth plays SoundFonts, which are widely available, and there
are tools for making your own SoundFonts from sample recordings.

For more sophisticated synthesis, the C library Csound, you can use it via
PInvoke. Csound is basically as powerful as it gets in sound synthesis.
Csound can use FluidSynth. Csound also has its own basic toolkit for simple
sample plaback, or you can build your own more complex samplers using
Csound's orchestra language.

Hope this helps,
Mike


-----------------------------------------------------
Michael Gogins
Irreducible Productions
http://michaelgogins.tumblr.com
Michael dot Gogins at gmail dot com

Post by Mark Garvin
I realize that this is slightly off the beaten path for this group,
I had written software for notation-based composition and playback of
orchestral scores. That was done via MIDI. I was working on porting
the original C++ to C#, and everything went well...except for playback.
The world has changed from MIDI-based rack-mount samplers to computer-
based samples played back via hosted VSTi's.
And unfortunately, hosting a VSTi is another world of involved software
development, even with unmanaged C++ code. Hosting with managed code
(C#) should be possible, but I don't think it has been done yet. So
I'm stuck. I've spoken to Marc Jacobi, who has a managed wrapper for
VST C++ code, but VSTi hosting is still not that simple. Marc is very
helpful and generous, and I pester him once a year, but it remains an
elusive problem.
It occurred to me that one of the resourceful people here may have
ideas for working around this. What I'm looking for, short term, is
simply a way to play back orchestral samples or even guitar/bass/drums
as a way of testing my ported C# code. Ideally send note-on, velocity,
note-off, similar to primitive MIDI. Continuous controller for volume
would be icing.
Any ideas, however abstract, would be greatly appreciated.
MG
NYC
--
subscription info, FAQ, source code archive, list archive, book reviews,
dsp links
http://music.columbia.edu/cmc/music-dsp
http://music.columbia.edu/mailman/listinfo/music-dsp

[Ross Bencina]

Post by Michael Gogins
For straight sample playback, the C library FluidSynth, you can use it via
PInvoke. FluidSynth plays SoundFonts, which are widely available, and there
are tools for making your own SoundFonts from sample recordings.
For more sophisticated synthesis, the C library Csound, you can use it via
PInvoke. Csound is basically as powerful as it gets in sound synthesis.
Csound can use FluidSynth. Csound also has its own basic toolkit for simple
sample plaback, or you can build your own more complex samplers using
Csound's orchestra language.

If I understand correctly the OP wants a way to host Kontakt and other
commercial sample players within a C# application, not to code his own
sample player or use something open source.

The question is the quickest path to hosting pre-existing VSTis in C#
and sending them MIDI events.

Ross.

[Michael Gogins]

Sorry for the misunderstanding.

I think the VSTHost code could be adapted. It is possible to mix managed
C++/CLI and unmanaged standard C++ code in a single binary. I think this
could be used to provide a .NET wrapper for the VSTHost classes that C#
could use.

Regards,
Mike


-----------------------------------------------------
Michael Gogins
Irreducible Productions
http://michaelgogins.tumblr.com
Michael dot Gogins at gmail dot com

Post by Ross Bencina

Post by Michael Gogins
For straight sample playback, the C library FluidSynth, you can use it via
PInvoke. FluidSynth plays SoundFonts, which are widely available, and there
are tools for making your own SoundFonts from sample recordings.
For more sophisticated synthesis, the C library Csound, you can use it via
PInvoke. Csound is basically as powerful as it gets in sound synthesis.
Csound can use FluidSynth. Csound also has its own basic toolkit for simple
sample plaback, or you can build your own more complex samplers using
Csound's orchestra language.

If I understand correctly the OP wants a way to host Kontakt and other
commercial sample players within a C# application, not to code his own
sample player or use something open source.
The question is the quickest path to hosting pre-existing VSTis in C# and
sending them MIDI events.
Ross.
--
subscription info, FAQ, source code archive, list archive, book reviews,
dsp links
http://music.columbia.edu/cmc/music-dsp
http://music.columbia.edu/mailman/listinfo/music-dsp

[Ross Bencina]

Post by Michael Gogins
I think the VSTHost code could be adapted. It is possible to mix managed
C++/CLI and unmanaged standard C++ code in a single binary. I think this
could be used to provide a .NET wrapper for the VSTHost classes that C#
could use.

I agree.

Maybe I missed something, but which VSTHost classes are you referring to?

Ross.