Is there an working example playing audio or videos?

Crank Storyboard Designer samples and code snippets.

Moderator: jamiev

Re: Is there an working example playing audio or videos?

Postby Miluba » Mon Nov 16, 2015 12:33 pm

Hi,
sry I've sent you the wrong commands ;-)
We are useing this
./gstreamer-backend -v -d3 -e -p "udpsrc port=5000 ! application/x-rtp, encoding-name=H264,payload=96 ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! xvimagesink ! appsink name=video-sink" & ./sbengine -vv -ogreio /home/unitylab/workspace_crank/VideoPlayer/VideoPlayer.gapp

Is it possible that there is a Problem, because I have no path where the a video file is?

I'm not shure if I understand your gstreamer-backend:
Where is the connection between the backend and Storyboard?
Is storyboard playing the stream or the backend?

best regards
Miluba
 
Posts: 7
Joined: Mon Oct 19, 2015 8:31 pm

Re: Is there an working example playing audio or videos?

Postby rdowdall » Mon Nov 16, 2015 12:42 pm

Hello Micheal

So change your gstreamer-backend pipeline to:

./gstreamer-backend -v -d3 -e -p "udpsrc port=5000 ! application/x-rtp, encoding-name=H264,payload=96 ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! video/x-raw-rgb,bpp=16,depth=16 ! appsink name=video-sink"

The xvimagesink that is in the line that you posted will consume the buffer and I don't think it will get passed along to the appsink.

There should be no problem in having a pipeline that does not have a file source as long as you don't specify a file name in the play.new.video action.

The way the gstreamer-backend works is that it decodes the stream and places it in an image buffer. The contents of that image buffer are placed in a shared memory region and then Storyboard engine is notified that new content is in available in the shared memory region. The Storyboard engine grabs the new content after this notification and places it on the screen.

So the gstreamer-backend is responsible for decoding the stream, using the gstreamer libraries, and then it sends the result of the decoding to Storyboard engine, which is responsible for displaying it on the screen.

Thanks,
Rodney
rdowdall
 
Posts: 114
Joined: Mon Nov 02, 2009 9:46 pm

Re: Is there an working example playing audio or videos?

Postby Miluba » Tue Nov 17, 2015 11:19 am

Hello,
we have still problems:

This is the Command for the Client:
./gstreamer-backend -v -d3 -e -p "udpsrc port=5000 ! application/x-rtp, encoding-name=H264,payload=96 ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! appsink name=video-sink"
& ./sbengine -vv -ogreio /home/unitylab/OrinocoIC_V2/VideoPlayer/VideoPlayer.gapp

This is our Test screen for the Server:
gst-launch-0.10 -v videotestsrc ! 'video/x-raw-rgb,width=800,height=400,bpp=16,depth=16,framerate=15/1' ! ffmpegcolorspace ! x264enc ! rtph264pay ! udpsink host=192.168.2.128 port=9078

We get not realy a picture (see Screenshot)

If we add this part to our Client command : video/x-raw-rgb,bpp=16,depth=16

We will get a pipeline Error.

Is it right that StoryBoard needs the rgb conversion, because I think the x264 is using yuv?
Can this be a problem? and have we only to change our pipeline for rgb conversion?

Best regards
Attachments
Screenshot from 2015-11-17 16_50_39.png
Screenshot from 2015-11-17 16_50_39.png (218.75 KiB) Viewed 29612 times
Miluba
 
Posts: 7
Joined: Mon Oct 19, 2015 8:31 pm

Re: Is there an working example playing audio or videos?

Postby rdowdall » Tue Nov 17, 2015 12:53 pm

Hello Michael

I assumed the video-convert element in the pipeline would be able to convert it to RGB from YUV. It's possible that is not the case. Can you try the following pipeline to see if that helps?

/gstreamer-backend -v -d3 -e -p "udpsrc port=5000 ! application/x-rtp, encoding-name=H264,payload=96 ! rtph264depay ! h264parse ! avdec_h264 ! ffmpegcolorspace ! video/x-raw-rgb,bpp=16,depth=16 ! appsink name=video-sink"

Your assumption about Storyboard needing an RGB buffer is correct. The x264 decoder is going to provide output that is in a YUV format, so it needs to be converted to an RGB format for Storyboard to show the video. A 16 bit RGB format is the best place to start. This is because with a 32 bit RGB format you need to specify a red mask, green mask and blue mask to get the colors to look correct in Storyboard.

Thanks,
Rodney
rdowdall
 
Posts: 114
Joined: Mon Nov 02, 2009 9:46 pm

Re: Is there an working example playing audio or videos?

Postby Ray » Mon May 16, 2016 1:17 pm

Hi, I am a newbie non-CS background software engineer trying to play video/audio on storyboard. (I am running storyboard both on Ubuntu 14.04 and Windows 10.)
It seems like Storyboard 4.2 has "gstramer-baecken" and "libgre-plugin-media.so" located in the correct directory on Linux. Now, when I try to run the content of "pipeline.txt"

./gstreamer-backend -v -d3 -e -p "filesrc location=blah name=media-src ! decodebin2 ! ffmpegcolorspace ! video/x-raw-rgb,bpp=16,depth=16 ! appsink name=video-sink"

what does "blah" imply?
And how to specify the full path to the gapp file? (For example, if the path of my gapp file is /home/roy/Workspace/VideoPlayer/Videplayer.gapp)

Thank you.

rdowdall wrote:Hello Michael

My co-workers and I put together a sample for you. It can be downloaded from here:

https://www.dropbox.com/s/41gh295aywt6l ... ar.gz?dl=0

The gstreamer-backend application should be placed in the same directory as the sbengine binary. The libgre-plugin-media.so library should be placed in the plugins directory of the runtime that you are using.

The gstreamer-backend application should be launched with the parameters that are specified in the file pipeline.txt. The VideoPlayer directory contains a Storyboard project that plays a video file. When you launch it, please specify the full path to the gapp file. This will help sbengine find the media file that is specified to play.

If you have any further questions, please let me know.

Thanks,
Rodney
Ray
 
Posts: 42
Joined: Wed Apr 06, 2016 3:32 pm

Re: Is there an working example playing audio or videos?

Postby rdowdall » Mon May 16, 2016 1:34 pm

Hello Ray

The blah is a placeholder. This value is going to get filled in by the gra.media.new.video action. This action will cause a media event to be sent to the gstreamer-backend application.

The full path to the gapp file needs to be specified when running Storyboard engine. This is needed so that the engine can resolve files correctly. So when you run sbengine you will want to do something like:

sbengine /home/roy/Workspace/VideoPlayer/Videoplayer.gapp

Are you launching this from the command line, or are you launching it using the simulator in Storyboard Designer?

Thanks,
Rodney
rdowdall
 
Posts: 114
Joined: Mon Nov 02, 2009 9:46 pm

Re: Is there an working example playing audio or videos?

Postby Ray » Tue May 17, 2016 8:28 am

Hi Rodney,

Thanks for the prompt reply. I am using Storyboard Designer to launch the example on Unbuntu14. I guess the path of gapp file has been specified correctly when I import the VideoPlayer project?
When I run the simulator after importing the VideoPlayer project, Console shows that

WARN [0.027]:Unable to attach to tslib device
WARN [0.885]:media: Can't open channel to backend media player [com.crank.media_backend] error 2

Do I have to execute gstreamer-backend from terminal before running the simulator?
Ray
 
Posts: 42
Joined: Wed Apr 06, 2016 3:32 pm

Re: Is there an working example playing audio or videos?

Postby rdowdall » Tue May 17, 2016 3:07 pm

Hello

Just to make sure that information is available to those that need to find it, Ray switched to posting in this topic after posting here:

viewtopic.php?t=476&p=1538#p1538

The response that was provided in that forum was:

"We don't currently have a media backend for Windows 10 that is released. We have been working on something in house that we will work on Windows 10 but no release timeline has been given for it. If you would like to obtain a pre-released version for Windows 10 then please contact our sales team.

For Ubuntu you can use the gstreamer-backend that is in the linux-x86-opengles_2.0-x11-obj/bin directory. This version of gstreamer-backend is built against gstreamer-0.10 so please make sure that the 32 bit versions of the gstreamer-0.10 libraries are installed on the Ubuntu system. There will be a version of the gstreamer-backend released in the future that is based on the gstreamer-1.0 libraries.

To use the gstreamer-backend application, just run it before you attempt to play media from the Storyboard application. It would probably be best to start the gstreamer-backend application before starting Storyboard engine. The LD_LIBRARY_PATH will have to be set to include the linux-x86-opengles_2.0-x11-obj/lib directory before starting gstreamer-backend. "

The key pieces of information in that post for playing media on Ubuntu are the last two paragraphs.

Thanks,
Rodney
rdowdall
 
Posts: 114
Joined: Mon Nov 02, 2009 9:46 pm

Previous

Return to Samples

Who is online

Users browsing this forum: No registered users and 2 guests