[Kdenlive-devel] max_analyze

Discussion:

[Kdenlive-devel] max_analyze_duration reached

Pascal Fleury

2014-09-22 00:30:12 UTC

Hello,

I am completely stuck, any help welcome...

I am about to finish a video project, where I use pretty long files
(several ***@25 13 minute chunks, audio @ 96kHz stereo 32 bit float for
1.5 hours) and it seemed to work nicely so far. I used proxy clips all
along, as working off the original video is just not possible.

Now I try to render the project, and I face many issues: first, when
rendering the project with proxy clips set on the clips (the little 'P'),
it does not use the full resolution for the final render. I think this is a
known issue (ignores the little checkbox in the render dialog).

My real issue is that if I remove the proxy clips, then it tries to
re-render the thumbnails on the time line, and gets them all wrong. The
files are correct, but it does not display the right images (takes some
others), and renders full black video.

After saving and reloading, it's even worse, as I face the issue with lots
of lines like in the title (more, see below). And after this, _all_ clips
on the timeline are zero length, completely ruining my project. I use git
for all text files, and did regular checkpoints, so I have not lost much
besides time, and rollbacks are easy.

So essentially, I am stuck with a project I cannot render after 60 hours of
editing time... Any help ? Anything I could do to get a final video from my
project ? Or is this max_analyze_duration from an underlying tool that I
could downgrade/upgrade ?

Also, I faced numerous issues with this project, mostly due to the size of
data and project I think, and I should probably do a write up. Where would
be a good avenue for this ? This list ?

Thanks,
Pascal

$ kdenlive
Fontconfig warning: "/etc/fonts/conf.d/50-user.conf", line 14: reading
configurations from ~/.fonts.conf is deprecated.
[wav @ 0x7fe2f45da760] max_analyze_duration reached
[wav @ 0x7fe2f45e2620] max_analyze_duration reached
[wav @ 0x7fe2f48549c0] max_analyze_duration reached
[wav @ 0x7fe2f462a360] max_analyze_duration reached
[wav @ 0x7fe2f4608b20] max_analyze_duration reached
[wav @ 0x7fe2f4613220] max_analyze_duration reached
[mpegts @ 0x7fe2f4909a40] max_analyze_duration reached
[mpegts @ 0x7fe2f490cd80] max_analyze_duration reached
[mpegts @ 0x7fe2f4c35680] max_analyze_duration reached
[wav @ 0x7fe2f49175e0] max_analyze_duration reached
[wav @ 0x7fe2f49196a0] max_analyze_duration reached
[mpegts @ 0x7fe2f4c24220] max_analyze_duration reached
[mpegts @ 0x7fe2f4c29c00] max_analyze_duration reached
[mpegts @ 0x7fe2f49bfb20] max_analyze_duration reached
[...and hundreds more...]

Dan Dennedy

2014-09-22 01:30:27 UTC

Permalink

Post by Pascal Fleury
Hello,
I am completely stuck, any help welcome...
I am about to finish a video project, where I use pretty long files
1.5 hours) and it seemed to work nicely so far. I used proxy clips all
along, as working off the original video is just not possible.

I never use proxy clips, almost always source clips, and very occasionally
transcode problematic clips.

Post by Pascal Fleury
Now I try to render the project, and I face many issues: first, when
rendering the project with proxy clips set on the clips (the little 'P'),
it does not use the full resolution for the final render. I think this is a
known issue (ignores the little checkbox in the render dialog).
My real issue is that if I remove the proxy clips, then it tries to
re-render the thumbnails on the time line, and gets them all wrong. The
files are correct, but it does not display the right images (takes some
others), and renders full black video.
After saving and reloading, it's even worse, as I face the issue with lots
of lines like in the title (more, see below). And after this, _all_ clips
on the timeline are zero length, completely ruining my project. I use git
for all text files, and did regular checkpoints, so I have not lost much
besides time, and rollbacks are easy.

Sounds like "remove the proxy clips" is not a safe operation.

Post by Pascal Fleury
So essentially, I am stuck with a project I cannot render after 60 hours
of editing time... Any help ? Anything I

I think you need a workaround for that problem with the checkbox being
ignored, but I do not know anything about the proxy mode in Kdenlive. Maybe
there is a way to modify the xml and render it with melt.

Post by Pascal Fleury
could do to get a final video from my project ? Or is this
max_analyze_duration from an underlying tool that I could downgrade/upgrade
?
I can definitely tell you that the message "max_analyze_duration reached"

comes from libavformat. It appears very frequently in many tools (too
verbose), is related to determining the duration of a clip, and is not
likely related to your problem.

Post by Pascal Fleury
Also, I faced numerous issues with this project, mostly due to the size of
data and project I think, and I should probably do a write up. Where would
be a good avenue for this ? This list ?
Thanks,
Pascal
$ kdenlive
Fontconfig warning: "/etc/fonts/conf.d/50-user.conf", line 14: reading
configurations from ~/.fonts.conf is deprecated.
[...and hundreds more...]

Vincent PINON

2014-09-22 08:27:15 UTC

Permalink

On Sun, Sep 21, 2014 at 5:30 PM, Pascal Fleury
Hello,

Hello Pascal,

I am completely stuck, any help welcome...
I am about to finish a video project, where I use pretty long
bit float for 1.5 hours) and it seemed to work nicely so far. I
used proxy clips all along, as working off the original video is
just not possible.
I never use proxy clips, almost always source clips, and very
occasionally transcode problematic clips.
Now I try to render the project, and I face many issues: first,
when rendering the project with proxy clips set on the clips (the
little 'P'), it does not use the full resolution for the final
render. I think this is a known issue (ignores the little checkbox
in the render dialog).

No, this is not an issue I have faced already (for more than 2 years).
Which version of Kdenlive (&MLT) are you using ?

My real issue is that if I remove the proxy clips, then it tries
to re-render the thumbnails on the time line, and gets them all
wrong. The files are correct, but it does not display the right
images (takes some others), and renders full black video.

Stupid question (but sometimes we need to open eyes when in panic ;-)
)... if source media are not on the same drive as project & proxies, is
that drive properly plugged & mounted ?

After saving and reloading, it's even worse, as I face the issue
with lots of lines like in the title (more, see below). And after
this, _all_ clips on the timeline are zero length, completely
ruining my project. I use git for all text files, and did regular
checkpoints, so I have not lost much besides time, and rollbacks
are easy.
Sounds like "remove the proxy clips" is not a safe operation.

Also never seen problems switching proxies on and off (but I don't use
thumbnails, except for testing)...

So essentially, I am stuck with a project I cannot render after 60
hours of editing time... Any help ? Anything I
I think you need a workaround for that problem with the checkbox being
ignored, but I do not know anything about the proxy mode in Kdenlive.
Maybe there is a way to modify the xml and render it with melt.

1) enable debug output (run "kdebugdialog" and check kdenlive)?
2) in render dialog, generate scripts.
for huge projects like yours, kdenlive eats much RAM, and melt will need
also much for full res rendering.
so closing kdenlive and rendering by launching the *.sh helps in general
(but your problems are unrelated, this is a side remark).
with scripts you get plain *.mlt files that the ones actually
transmitted to melt for rendering (ie, replacing proxies etc.)
so these are the files you can analyze to better understand your
rendering problem: try to diff *.mlt checking and unchecking the "use
proxy" option: of course the source media are changing, but all lengths
and playlists should remain the same...

could do to get a final video from my project ? Or is this
max_analyze_duration from an underlying tool that I could
downgrade/upgrade ?
I can definitely tell you that the message "max_analyze_duration
reached" comes from libavformat. It appears very frequently in many
tools (too verbose), is related to determining the duration of a clip,
and is not likely related to your problem.

yes, max_analyze_duration is coming all the time.

Also, I faced numerous issues with this project, mostly due to the
size of data and project I think, and I should probably do a write
up. Where would be a good avenue for this ? This list ?

Bug tracker...

Thanks,
Pascal

Hope we can find the solution for your problem...
Vincent

$ kdenlive
reading configurations from ~/.fonts.conf is deprecated.
[...and hundreds more...]

Pascal Fleury

2014-09-22 17:06:20 UTC

Permalink

Hi Vincent,

Post by Vincent PINON

On Sun, Sep 21, 2014 at 5:30 PM, Pascal Fleury
Hello,

Hello Pascal,

No, this is not an issue I have faced already (for more than 2 years).
Which version of Kdenlive (&MLT) are you using ?

I am using kdenlive v0.9.8 on ubuntu, from this PPA
http://ppa.launchpad.net/sunab/kdenlive-release/ubuntu/
mlt v 0.9.0
frei0r-plugins 1.4.0 (probably irrelevant)

Post by Vincent PINON

Stupid question (but sometimes we need to open eyes when in panic ;-)
)... if source media are not on the same drive as project & proxies, is
that drive properly plugged & mounted ?

All the files are on the same SATA 3TB drive, on ext4 :-)

Post by Vincent PINON

Also never seen problems switching proxies on and off (but I don't use
thumbnails, except for testing)...

As I told Dan, I wonder if this is related to the tool not being able to
analyze the length of a file, then defaulting to 0, then modifying the
clips on the timeline as a result of the length being 0.

Post by Vincent PINON

I will try the mlt script approach. but I don't think RAM is an issue. I
have rendered some shorter videos at the same resolution, using different

Post by Vincent PINON

yes, max_analyze_duration is coming all the time.

Also, I faced numerous issues with this project, mostly due to the
size of data and project I think, and I should probably do a write
up. Where would be a good avenue for this ? This list ?

Bug tracker...

Ok, will file some more bugs (I filed a few in the past).
The write up will also mention things like audio getting out of sync, or
being shifted in the rendered movie and not in the editor. and with the
latest version (0.9.8), I notices very surprising things, like the play
rate of audio only tracks to change with time. Like in th egood old LP
times when you move the selector from 45 to 33rpm...

Post by Vincent PINON

Thanks,
Pascal

Hope we can find the solution for your problem...

I'm comfortable compiling from source and running that. What is the current
branch I should use ?

Post by Vincent PINON
Vincent

$ kdenlive
reading configurations from ~/.fonts.conf is deprecated.
[...and hundreds more...]

------------------------------------------------------------------------------
Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk
_______________________________________________
Kdenlive-devel mailing list
https://lists.sourceforge.net/lists/listinfo/kdenlive-devel

Vincent Pinon

2014-09-22 20:35:24 UTC

Permalink

Post by Pascal Fleury
I am using kdenlive v0.9.8 on ubuntu, from this PPA
http://ppa.launchpad.net/sunab/kdenlive-release/ubuntu/
mlt v 0.9.0

Sorry you are on release track, so not affected by the patch I just mentioned

Post by Pascal Fleury
I will try the mlt script approach. but I don't think RAM is an issue. I
have rendered some shorter videos at the same resolution, using different

I've seen RAM being eaten by projects with many small clips (few seconds cuts
for >30mn total), but if your media are large files maybe you are less affected.

Post by Pascal Fleury
Ok, will file some more bugs (I filed a few in the past).
The write up will also mention things like audio getting out of sync, or
being shifted in the rendered movie and not in the editor. and with the
latest version (0.9.8), I notices very surprising things, like the play
rate of audio only tracks to change with time. Like in th egood old LP
times when you move the selector from 45 to 33rpm...

Seen this once but didn't have time to identify the cause. It was format
dependent and transcoding helped...

Post by Pascal Fleury
I'm comfortable compiling from source and running that. What is the current
branch I should use ?

master
but if you already use sunab's PPA you have a -development directory (or so)
with daily builds, that saves you this effort.

Pascal Fleury

2014-09-22 17:05:43 UTC

Permalink

Hi Dan,

On Sun, Sep 21, 2014 at 5:30 PM, Pascal Fleury <

Post by Pascal Fleury
Hello,
I am completely stuck, any help welcome...
I am about to finish a video project, where I use pretty long files
for 1.5 hours) and it seemed to work nicely so far. I used proxy clips all
along, as working off the original video is just not possible.

I never use proxy clips, almost always source clips, and very occasionally
transcode problematic clips.

On my machine, navigating through original clips is way too slow. Also I
have 4 video tracks, with mostly 2 tracks having a video at any given time.
Proxy clips are actually making the whole thing very usable, if not really
comfortable.

Post by Pascal Fleury
Now I try to render the project, and I face many issues: first, when
rendering the project with proxy clips set on the clips (the little 'P'),
it does not use the full resolution for the final render. I think this is a
known issue (ignores the little checkbox in the render dialog).
My real issue is that if I remove the proxy clips, then it tries to
re-render the thumbnails on the time line, and gets them all wrong. The
files are correct, but it does not display the right images (takes some
others), and renders full black video.
After saving and reloading, it's even worse, as I face the issue with
lots of lines like in the title (more, see below). And after this, _all_
clips on the timeline are zero length, completely ruining my project. I use
git for all text files, and did regular checkpoints, so I have not lost
much besides time, and rollbacks are easy.

Sounds like "remove the proxy clips" is not a safe operation.

I think it tries to get the original clip length, and from the error
message, seems to not get a useful answer. If it returns a length of zero
(or negative) in case it reaches the maximum analysis length, then it would
explain why my clips get all shrunk on the timeline.

Post by Pascal Fleury
So essentially, I am stuck with a project I cannot render after 60 hours
of editing time... Any help ? Anything I

I will fiddle a bit more with that checkbox, as I think I had other issues
with sub-projects that I imported.

reached" comes from libavformat. It appears very frequently in many tools
(too verbose), is related to determining the duration of a clip, and is not
likely related to your problem.

As I mentioned above, does libavformat then return a size of e.g. 0 if it
could not analyze it fully ?
Is there a cmdline tool that I could run on these WAV and video files to
see the failure mode ?

Post by Pascal Fleury
Also, I faced numerous issues with this project, mostly due to the size
of data and project I think, and I should probably do a write up. Where
would be a good avenue for this ? This list ?
Thanks,
Pascal
$ kdenlive
Fontconfig warning: "/etc/fonts/conf.d/50-user.conf", line 14: reading
configurations from ~/.fonts.conf is deprecated.
[...and hundreds more...]

Dan Dennedy

2014-09-22 18:27:28 UTC

Permalink

On Mon, Sep 22, 2014 at 10:05 AM, Pascal Fleury <

Post by Pascal Fleury
Hi Dan,

On Sun, Sep 21, 2014 at 5:30 PM, Pascal Fleury <

I never use proxy clips, almost always source clips, and very
occasionally transcode problematic clips.

Post by Pascal Fleury
Now I try to render the project, and I face many issues: first, when
rendering the project with proxy clips set on the clips (the little 'P'),
it does not use the full resolution for the final render. I think this is a
known issue (ignores the little checkbox in the render dialog).
My real issue is that if I remove the proxy clips, then it tries to
re-render the thumbnails on the time line, and gets them all wrong. The
files are correct, but it does not display the right images (takes some
others), and renders full black video.
After saving and reloading, it's even worse, as I face the issue with
lots of lines like in the title (more, see below). And after this, _all_
clips on the timeline are zero length, completely ruining my project. I use
git for all text files, and did regular checkpoints, so I have not lost
much besides time, and rollbacks are easy.

Sounds like "remove the proxy clips" is not a safe operation.

Post by Pascal Fleury
So essentially, I am stuck with a project I cannot render after 60 hours
of editing time... Any help ? Anything I

I will fiddle a bit more with that checkbox, as I think I had other issues
with sub-projects that I imported.

reached" comes from libavformat. It appears very frequently in many tools
(too verbose), is related to determining the duration of a clip, and is not
likely related to your problem.

As I mentioned above, does libavformat then return a size of e.g. 0 if it
could not analyze it fully ?

I just looked at libavformat code to learn more about this. It is not only
determining the duration; it is gathering all the essential information
about the file like tracks, codecs, fps, initial timestamps, metadata, etc.
Depending upon the format, sometimes it is not clear when all of the
information has been collected. In that case it reads frames/packets
representing up to max_analyze_duration (5s default), to finish collecting
more possible information. More often than not, it will have collected it,
but it is being diligent and reporting verbosely about its diligence.

Post by Pascal Fleury
Is there a cmdline tool that I could run on these WAV and video files to
see the failure mode ?

melt -verbose {filename} -consumer xml time_format=clock

Then, look for the "length" property (HH:MM:SS.ms) to see if it seems
correct. It also reports some information about streams and codecs so you
can determine if that looks sane.

--
+-DRD-+

Vincent Pinon

2014-09-22 20:27:04 UTC

Permalink

Post by Pascal Fleury
I will fiddle a bit more with that checkbox, as I think I had other issues
with sub-projects that I imported.

Ah, you are importing kdenlive projects! That's the point.
A kdenlive project is a MLT playlist, but if you use proxies in subproject
then those are the ones called in the playlist (referring to original source
only in kdenlive "private" area).
So either you have to disable proxies in subproject, or I prefer to keep them
as is, and generate a plain and clean MLT playlist through Renderer / Generate
script, and pick *.mlt in scripts/ folder (it could be possible to also create
a XML output profile).

Then I suspect the wrong file length are in the master project for these
subprojects?
We have a bug when we mix projects framerates (#3320), and trying to fix it, we
had a patch during august that messed producers length for everything else
than videos @ integer fps (leading to random/large/negative length).
I tried to refine the correction several times and finally just reverted to the
original state as it seems the less bad solution for the moment!

To know if you are concerned, you need to check the package date, as
unfortunately I removed git builds tags from dev versions (triggering large
rebuilds for every commit, slowing development process on my poor hardware).

Good luck!

Vincent

Evert Vorster

2014-09-22 09:01:34 UTC

Permalink

Post by Pascal Fleury
Hello,
I am completely stuck, any help welcome...

Let's see if I can shed some light on this...

Post by Pascal Fleury
I am about to finish a video project, where I use pretty long files (several
and it seemed to work nicely so far. I used proxy clips all along, as
working off the original video is just not possible.
Now I try to render the project, and I face many issues: first, when
rendering the project with proxy clips set on the clips (the little 'P'), it
does not use the full resolution for the final render. I think this is a
known issue (ignores the little checkbox in the render dialog).

Look at issue 0003337 in the mantis bug tracker. This bug report has
very similar symptoms to what is described here.
That bug has been squashed only in the last month. That means that
there is no released version of kdenlive that will fix this issue.

If you have not done so already, fetch the latest git of mlt and
kdenlive, build and install those, and see if your issue pesists.

Post by Pascal Fleury
My real issue is that if I remove the proxy clips, then it tries to
re-render the thumbnails on the time line, and gets them all wrong. The
files are correct, but it does not display the right images (takes some
others), and renders full black video.

This also sounds like an old bug that has been fixed in the latest
git, and unrelated to the rendering from floxies bug.

Post by Pascal Fleury
After saving and reloading, it's even worse, as I face the issue with lots
of lines like in the title (more, see below). And after this, _all_ clips on
the timeline are zero length, completely ruining my project. I use git for
all text files, and did regular checkpoints, so I have not lost much besides
time, and rollbacks are easy.

This is interesting. I would like to know more about how you do this,
as I have also lost considerable amount of time when a save file got
corrupted.

Post by Pascal Fleury
So essentially, I am stuck with a project I cannot render after 60 hours of
editing time... Any help ? Anything I could do to get a final video from my
project ? Or is this max_analyze_duration from an underlying tool that I
could downgrade/upgrade ?

Fetch the latest git of kdenlive and mlt, and install those. Then load
your project. You might have to delete the thumbnails to get kdenlive
to re-render them.
Also, if you disable proxies for the project, and save as a different
project, you might have some success that way.... however, proxy
handling has been greatly improved in the last month or so in
kdenlive.

Check & report bugs on
http://www.kdenlive.org/mantis/my_view_page.php

Kind regards,
Evert Vorster

Pascal Fleury

2014-09-22 17:06:43 UTC

Permalink

Hi Evert,

Post by Evert Vorster

Post by Pascal Fleury
Hello,
I am completely stuck, any help welcome...

Let's see if I can shed some light on this...

Post by Pascal Fleury
I am about to finish a video project, where I use pretty long files

(several
hours)

Post by Pascal Fleury
and it seemed to work nicely so far. I used proxy clips all along, as
working off the original video is just not possible.
Now I try to render the project, and I face many issues: first, when
rendering the project with proxy clips set on the clips (the little

'P'), it

Post by Pascal Fleury
does not use the full resolution for the final render. I think this is a
known issue (ignores the little checkbox in the render dialog).

Look at issue 0003337 in the mantis bug tracker. This bug report has
very similar symptoms to what is described here.
That bug has been squashed only in the last month. That means that
there is no released version of kdenlive that will fix this issue.
If you have not done so already, fetch the latest git of mlt and
kdenlive, build and install those, and see if your issue pesists.

This also sounds like an old bug that has been fixed in the latest
git, and unrelated to the rendering from floxies bug.

Post by Pascal Fleury
After saving and reloading, it's even worse, as I face the issue with

lots

Post by Pascal Fleury
of lines like in the title (more, see below). And after this, _all_

clips on

Post by Pascal Fleury
the timeline are zero length, completely ruining my project. I use git

for

Post by Pascal Fleury
all text files, and did regular checkpoints, so I have not lost much

besides

Post by Pascal Fleury
time, and rollbacks are easy.

This is interesting. I would like to know more about how you do this,
as I have also lost considerable amount of time when a save file got
corrupted.

Well, in a previous project, I spent about 10 hours recovering from a
timeline corruption (selecting a lot of clips on the timeline, and shifting
them would just completely messup things). After that, I used git to track
all text files (none of the videos though). That includes titles, video
stabilization output files, etc.

I almost never use any merge/patch for these XML files, as some simple
edits have quite a big effect on the XML (and it's too easy to get into an
even more corrupted state :-). However, git has very cheap branching, and
very cheap commits, so whenever I have my project in a state I think is
fine, I do a git commit. So I can go back to that point. And I do it each
time I finish a session.

This is definitely not a tool for Joe User, but I am a software engineer
and use git daily at work :-)

Post by Evert Vorster

Post by Pascal Fleury
So essentially, I am stuck with a project I cannot render after 60 hours

Post by Pascal Fleury
editing time... Any help ? Anything I could do to get a final video from

Post by Pascal Fleury
project ? Or is this max_analyze_duration from an underlying tool that I
could downgrade/upgrade ?

Ok, that is a good point, and will try compiling from source. Thanks.

--paf

Post by Evert Vorster
Check & report bugs on
http://www.kdenlive.org/mantis/my_view_page.php
Kind regards,
Evert Vorster
------------------------------------------------------------------------------
Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk
_______________________________________________
Kdenlive-devel mailing list
https://lists.sourceforge.net/lists/listinfo/kdenlive-devel