Page 1 of 2

Play a mp4 video on embedded linux

PostPosted: Thu Nov 01, 2018 1:45 am
by damon.feng
Hello,

I am new to gstreamer and I'd like to play videos on embedded linux environment. Storyboard tutorial told me that I can use gst-launch to play a video on my embedded linux environment before actually playing videos in Storyboard target application. My video's format is mp4 and encoding is h264. I can play it on my Linux PC environment with a command: gst-launch filesrc location=xxx.mp4 name=media-src ! qtdemux ! avdec_h264 ! autovideosink
However, when I tried this operation on my embedded linux environment, I failed. The command is gst-launch filesrc location=xxx.mp4 name=media-src ! qtdemux ! ffdec_h264 ! fbdevsink
My target board uses TI Am437x and version of gst-launch is 0.1. The version of gst-launch on my linxu pc is 1.0. The following screenshot shows the error I met. General stream error.
Capture2.JPG
Capture2.JPG (43.34 KiB) Viewed 202 times


Could someone help with my issue, please?

Re: Play a mp4 video on embedded linux

PostPosted: Thu Nov 01, 2018 12:11 pm
by rdowdall
Hello

The issue is probably caused by the fact that the ffdec_h264 decoder can't output a frame as something that the fbdevsink can understand. For example the ffdec_h264 decoder is probably outputting frames as YUV frames, and the fbdevsink probably only understands RGB content. You will need to add in a converter between the ffdec_h264 decoder and the fbdev sink. Usually this converter is called ffmpegcolorspace conveter for the 0.1 version of Gstreamer. So your pipeline would look something like this:

gst-launch filesrc location=/opt/tco/UI_LCD_GUI/movies/Start_Cleaning.mp4 name=media-src ! qtdemux ! ffdec_h264 ! ffmpegcolorspace ! fbdevsink

The other thing that may come in to play with this is that you map need to set the caps for what the ffmpegcolorspace converter is outputting as, so you would need to add something like this:

gst-launch filesrc location=/opt/tco/UI_LCD_GUI/movies/Start_Cleaning.mp4 name=media-src ! qtdemux ! ffdec_h264 ! ffmpegcolorspace ! video/x-raw-rgb,depth=32 ! fbdevsink

Thanks,
Rodney

Re: Play a mp4 video on embedded linux

PostPosted: Thu Nov 01, 2018 8:40 pm
by damon.feng
Hello Rdowdall,

Thank you for your answer. Unfortunately, I still failed. The following is my screenshots.
Capture3.JPG
Capture3.JPG (31.18 KiB) Viewed 186 times

Capture4.JPG
Capture4.JPG (27.29 KiB) Viewed 186 times

Re: Play a mp4 video on embedded linux

PostPosted: Fri Nov 02, 2018 3:43 am
by garry
Hi,

can you increase the information level for the trace output to show more details of why the pipeline is failing and share this. Checking the documentation for gst-launch (here is for 1.0 but it should be the same for 0.1, see https://gstreamer.freedesktop.org/docum ... aunch.html) you should be able to do this by adding the '-v' option for verbose on the pipeline command.

Please can you capture the output with your file and pipeline and post it so we can take a look.

Regards,
Garry

Re: Play a mp4 video on embedded linux

PostPosted: Fri Nov 02, 2018 8:39 am
by rdowdall
Hello

Garry's post is a good one to follow for getting extra information out of Gstreamer for the purpose of debugging issues with pipelines. Another tool that you can use is to run:

export GST_DEBUG=*:3

before running gst-launch. This requires that your gstreamer libraries though were built with the --enable-gst-debug flag turned on.

I would like to point out that the first pipeline you tried worked. It did not report any errors and it started the video. If you didn't see anything on the screen, then it could be because the processing of the video is extremely slow. Or it could be that the fbdevsink is pointing to a framebuffer device that is not hooked up to an output. Those are things that you would need to check on your system to make sure that things are configured properely. Can you run anything else and have it output to /dev/fb0?

Thanks,
Rodney

Re: Play a mp4 video on embedded linux

PostPosted: Fri Nov 02, 2018 9:25 pm
by damon.feng
Rdowdall & Garry,

Thank you for your suggestions! However, the root cause is I displayed my GUI on /dev/fb1 and default display device for fbdevsink seems /dev/fb0. Fb1 covered Fb0 so that I couldn't see the video when it was playing. Now I added property device=/dev/fb1 and the video can be seen.
My next step is to implement a feature of playing videos in my Storyboard application with gstreamer-backend. Per the tutorial, I think I just need to set correct media action to do this. Could you tell me how to set the parameters and prepare some default configuration if needed, please? Such as environment variables to tell the storyboard application where the gstreamer-backend is, channel name, ....

BTW, though I can play videos with gst-launch, but videos go very slowly. I think I must add some other parameters.

Re: Play a mp4 video on embedded linux

PostPosted: Mon Nov 05, 2018 12:29 pm
by rdowdall
Hello

This section of the documentation talks about how to setup the gstreamer backend pipeline:

https://www.cranksoftware.com/cranksoft ... 1794216800

If you are only going to be playing one type of file, then settting the pipeline on the gstreamer backend commandline is the way to go. If you are going to be playing different types of files, then setting the pipeline in the new.video action is recommended.

You can import the media sample that is provided with Storyboard Designer (for 5.0 versions and later) and start to play around with the new.video action that is in that sample.

For the slowness, this is probably a result of the color conversion from YUV to RGB. If your hardware has a color conversion unit on it, like an IPU, then there may be a gstreamer element that you can use to offload the color conversion to it, rather than using the CPU. Those are typically proprietary elements though that come from the hardware manufacturer.

You can try using the quality of service settings on the filesrc and video sink elements. Also there is sometimes a buffer option that can be set on the video sink that allows it to avoid memory copies and that will speed things up. If you pass along the output for gst-inspect on the fbdevsink, ffdev_h264, and fbdevsink elements, we can take a look to see if there are some options you can take advantage of, but it will depend on what is available.

Thanks,
Rodney

Re: Play a mp4 video on embedded linux

PostPosted: Wed Nov 07, 2018 1:50 am
by damon.feng
Hi Rdowdall,

I executed the following commands, but there's no feedback. Is my command correct?
Capture5.JPG
Capture5.JPG (28 KiB) Viewed 142 times


In addition, the second attachment is the output of my gst-inpsect. Could you help me check which element I can use to solve the slowness, please?

Re: Play a mp4 video on embedded linux

PostPosted: Fri Nov 09, 2018 8:02 am
by rdowdall
damon.feng wrote:Hi Rdowdall,

I executed the following commands, but there's no feedback. Is my command correct?
Capture5.JPG


In addition, the second attachment is the output of my gst-inpsect. Could you help me check which element I can use to solve the slowness, please?


Hello Damon

You need to place quotes around your pipeline. It won't parse the line correctly.

With respect to optimizing your pipeline, you can try the following:

  • On the filesrc element add the mmap=true option
  • On the ffdec_h264 element add the max-threads=4 option ( you can play around with this value )
  • On the fbdevsink element add the qos=true option

These may work for your board, but there is no guarantee that they will. When constructing a gstreamer pipeline it's about trying different options to see how they work on your particular hardware. You can experiment with gst-launch, as the gstreamer-backend application is similar to gst-launch, but has added functionality in it to talk to a Storyborad application.

Also you will probably want to do:

gst-inspect | grep converter

to see if there is another color space converter that you can use other than the software one.

Thanks,
Rodney

Re: Play a mp4 video on embedded linux

PostPosted: Mon Nov 12, 2018 1:19 am
by damon.feng
Hi Rdowdall,

Thank you for your reply!

I've already found the root cause of slowness, but I cannot fix it. The TI processor AM4376 of my development board doesn't have a hardware video decoder and its frequency is only up to 1GHz. Therefore, the processor cannot play back videos with h264 encoding and resolution 1280x720 smoothly and fluently. I converted the original video to MPEG4 and changed its resolution to 1024x576. The result is much better.

Unfortunately, I still cannot play back the MPEG4 video I created with gstreamer-backend. No any output. Please help me check this. It's too frustrating.
Capture7.JPG
Capture7.JPG (27.46 KiB) Viewed 8 times