I'm the owner of Broadcastify.com, where presumably these streams are being transcribed from. We've dabbled in this space and looked at real-world approaches to taking something like this to market, but transcribing 7000+ streams to text seems like an expensive (computational) and ($$) effort that needs a lot of investigation.
Note to mention that the individual lexicons between streams are drastically different.
I wonder how the developer has done the integration to our streams... because I never heard from them :)
My plan was to collect user transcription corrections on my site then train my own inexpensive models on them. The open-source speech tech I work on can do passable transcription at close to 100x faster than realtime on a quad core desktop CPU (or 200 simultaneous streams per 4-core box at 50% activity). With higher quality transcription it's closer to 10-20x faster than realtime.
For your case you could also try to push some of the computation down to the uploading machine. These models can run on a raspberry pi.
I think the biggest work for a new effort here is going to be building local language models and collecting transcribed audio to train on. However, there have been a couple of incredible advances in the last year for semi-supervised speech recognition learning, where we can probably leverage your 1 year backlog as "unsupervised training data" while only having a small portion of it properly transcribed.
The current state-of-the-art paper uses around 100 hours of transcribed audio and 60,000 hours of unlabeled audio, and I bet you could push the 100h requirement down with a good language model and mixing in existing training data from non-radio sources.
We're working on client ingest models now that work on more of a "tasking" perspective, where someone deploys a device that is GPS enabled and then we send an ingest task to fill in coverage, start new coverage, etc. But this is predicated on low cost ingest devices (read: RPi and RTL sticks) which might not have the horsepower needed for transcription at the client level.
https://rogueamoeba.com/loopback/
Someone clever enough could create containers to run the software locally and have many loops running off many streams to many instances of the audio to text feature.
I'd have to imagine that stream listening follows some sort of power (or otherwise 80/20) law, so hopefully that would help with the expense?
https://wiki.radioreference.com/index.php/Broadcastify-Calls
I wonder if one could mix in openstreetmap data for a location to help pick up local references. (Eventually would be cool to round trip it with a little ping when addresses/businesses are referenced).
Any thoughts on adding the ability to comment/transcribe/etc?
I selected the NYC scanner and found many examples like this:
June 8, 2020 9:03 PM EDT: "Google Launcher new job and I want to play better third-party colder or does the people from the vegetable okay"
> Please don't comment on whether someone read an article. "Did you even read the article? It mentions that" can be shortened to "The article mentions that."
Yes, I looked the website, and I also have my own website that does the same thing in a very similar manner. My comment was entirely in response to the suggestion Mac speech recognition be used for this. It should not be used for this. Based on previous experiments I have personally performed, it would be even worse than the website's accuracy. I then pointed out what a good solution might look like (and neither my website nor the linked website do the good solution yet)
We decided to ask for forgiveness on this project - we do in fact have a premium account with you guys! ffmpeg records and segments your streams, sox removes silence from scanners, Google transcribes, redis serves it all up quickly!
A broadcastify.com premium account is not not a great excuse, but I would love to have a conversation at length with you and your team! How do we get in contact? Thanks again!
Your prototype is amazing! The quality of transcription is definitely better than ours via Google.
After we did some legal research we wanted to avoid storing the recordings and rather solely transcription text. Giving access to a platform for humans to verify the transcriptions and in turn train the model is a great idea.
I have started working on getting some pre-trained models set up. I am trying to implement them with wav2letter, deepspeech, kaldi, vosk, etc. - I just need to be pointed in the right direction.
Raspberry Pi's were something I was considering as well - small energy footprint and powerful enough to run these models.
Do you have any advice on ML or acoustic models to avoid? I am working with the 100 hour dataset now.
Thanks!
I’ve had some decent results with the following:
I have to research how to hand tag my own samples to see if that offers significant accuracy improvements (let’s say I want to accurately transcribe one voice consistently).
Google and Watson APIs are not too free, and I believe Watson has a audio length limit (possibly limited by free tier, or possibly limited in general for all tiers).
Cool to see some real world attempts using this stuff.
I have 30ish streams and keep 6 days worth, I could keep longer if you'd like to work together on this. I reached out to some of the people above, the Broadcastify guy for example, and they are, as mentioned, ready doing their own thing so didn't really care about what I wanted to share.
I will say though I recently noticed that during the recent unrest in my city that the stream had a caption along the lines of 'police are using encrypted channels' which is understandable but disappointing from the perspective of a citizen looking for transparency.
I realize both the possible legal and technical difficulty of implementing something like this but have you had any conversations about how to maybe combat this? Times of unrest are not only a massive opportunity for Broadcastify to grow its user base but it's also when transparency is at its peak importance.