Jokosher Network Instruments

Jokosher needs to support instruments that can listen over the network. Because we are based on GStreamer, this opens up a world of possibilities for gstreamer audio in Jokosher where it is recorded. First lets talk use cases:

USE CASES

  • Bob records a really cool podcast, and he wants to interview with people over Voice Over IP. Ideally, he wants to do the interview, hear their audio through Jokosher and record it in his Jokosher project. Likewise, Bob wants his recorded audio to be streamed out to the listener.
  • Dave is recording a song and wants to get a bass line from John who is on holiday in the USA. John borrows a bass, and records his piece which is then recorded in Jokosher.

Infrastructure

(We are still designing much of the infrastructure now, but this is our thoughts)

You have a few different components here:

  • Jokosher Network Instruments - these are instruments that basically read their audio data from a network GStreamer element. This will be a src of some description which goes into the Instrument like anything else.
  • Source Server - something needs to push audio to the Jokosher Network Instrument, and a server would do this. So, if you want to record a VoIP conversation, you would need the server to be able to output audio to the Jokosher Network Instrument.

A few notes about each:

Jokosher Network Instruments

  • There will be a base kind of network instrument, but we will have specific instruments. So, you would have an Internet Phone instrument which could use Telepathy to list your contacts to determine who you call.
  • We need to think about what kind of data the network instrument accepts.
  • There needs to be a status light to indicate when network traffic is flowing.
  • Network instruments need to be able to send certain instruments from Jokosher to the server in certain cases. E.g. in a VoIP use case, the guest on the show needs to hear the presenters in the studio who are recording locally into Jokosher.

Servers

  • Servers can take in audio in any way they like (such as VoIP audio with SIP/IAX etc.), but they need to stream audio to Jokosher in the format the Jokoshet Network Instrument understands.
  • The server will need to stream each channel seperately. So, if you have a VoIP server with two guests, each guest needs to be streamed to a seperate Jokosher Nework Instrument so that Jokosher can treat them as seperate instruments.

Recording VoIP conversations

Jokosher can hook in with the Telepathy framework and Galago to make voice calling easy. The process will be as follows:

  • Start jokosher
  • Add a Vocal instrument to Jokosher
  • In Jokosher, do Edit > Instrument Connections, and tick "send to the network" next to Vocal
  • In Jokosher, add a "VoIP caller" instrument and select Alice from the 'online contacts' (populated via dbus from Galago)
  • In Jokosher, add a "VoIP caller" instrument and select Bob from the 'online contacts' (populated via dbus from Galago)
  • In Jokosher, press "Record"

(There is an issue with how you initiate the call; pressing Record is a simple way, but that means that you can't call without recording, to test mics, etc. The call is placed with Telepathy. See the irc discussion for more.)