Commit graph

2380 commits

Author SHA1 Message Date
Arttu Ylä-Outinen 7f7844caad Fix finalizing uninitialized encoder states
Finalization functions for frame and tile encoder states accessed the
frame and tile fields of the encoder state even though they might be
NULL. This is the case when the initialization of an encoder state
fails. Fixed by adding NULL checks.
2017-02-09 14:05:28 +09:00
Arttu Ylä-Outinen 51786eda67 Drop redundant fields in encoder_control_t
Some of the fields in encoder_control_t were simply copies of the
corresponding fields in kvz_config. This commit drops the copied fields
in favor of using the fields in encoder_control_t.cfg directly.
2017-02-09 14:05:28 +09:00
Arttu Ylä-Outinen 6a178dee96 Fix leaking memory when --cqmfile given many times
Any previously allocated CQM file name was not freed when allocating
memory for the new file name.
2017-02-09 14:05:28 +09:00
Arttu Ylä-Outinen 63a567ad8a Fix leaking memory when --roi given many times
Any previously allocated delta QP array was not freed when allocating
a new array.
2017-02-09 14:05:21 +09:00
Arttu Ylä-Outinen bfd89136a4 Fix ROI delta QP array not getting freed 2017-02-09 13:23:55 +09:00
Arttu Ylä-Outinen e78a8dfcf5 Copy the kvz_config passed to encoder_open
The kvz_config struct is created by the user but kvazaar keeps a pointer
to it. It is easy to break things by modifying the configuration outside
kvazaar. In addition, kvazaar modifies the struct even though it is has
a const modifier.

This commit changes the field cfg in encoder_control_t to be a copy of
the kvz_config struct instead of a pointer, removing modifications to
the const struct and allowing users to do whatever they want with it
after opening the encoder.
2017-02-09 13:23:54 +09:00
Ari Koivula 4b2d1c2686 Add tests for weird shapes. 2017-02-08 21:41:37 +02:00
Ari Koivula b8e3513a23 Fix crash with sub-LCU frame sizes and WPP
The end of slice was being calculated incorrectly, which led to no tile
being created inside the slice, which led to an assert triggering.

This fixes the wrong end of slice calculation, but also disallows
wavefront rows from being created, if there would be only one.
The wavefront initialization code assumes there are always more than
one row, so the inter-frame dependency doesn't get added properly.

Fixes #153.
2017-02-08 21:41:30 +02:00
Ari Koivula a4a3b0496b Update OS X build instructions 2017-02-03 18:34:28 +02:00
Ari Koivula d893474bab Fix encoder getting stuck on OS-X
Main thread was stuck looping on pthread_cond_timedwait because
the abs time given on OS-X had already passed and the wait
returned immediately without releasing the mutex to allow worker
threads to proceed.

Fix was to use the gettimeofday, which returns real time instead
of monotonic, which is what pthread_cond_timedwait wants.
2017-02-02 17:27:46 +02:00
Ari Koivula 4ceda1908b Fix OS-X compiler warning
rdo.c:475:25: warning: absolute value function 'abs' given an argument of type 'int64_t' (aka 'long long') but has parameter
       of type 'int' which may cause truncation of value [-Wabsolute-value]
         current.cost = -abs(quant_cost_in_bits) + (bits << PRECISION_INC);
                         ^
rdo.c:475:25: note: use function 'llabs' instead
         current.cost = -abs(quant_cost_in_bits) + (bits << PRECISION_INC);
2017-02-01 18:09:17 +02:00
Ari Koivula c7d536bbcd Fix OS-X compiler warning
cfg.c:1024:74: warning: format specifies type 'size_t' (aka 'unsigned
long') but the argument has type 'unsigned long long'
       [-Wformat]
       fprintf(stderr, "Too large ROI size: %llu (maximum %zu).\n", size, SIZE_MAX);
2017-02-01 18:09:04 +02:00
Ari Koivula 4467506ef1 Add missing kvz_ prefix 2017-01-31 18:38:02 +02:00
Ari Koivula ed3bd898fd Remove Exp-Golomb lookup table
This table takes 256kB and isn't used very much. Au revoir!
2017-01-31 18:31:05 +02:00
Ari Koivula e85266efe7 Update docs 2017-01-31 16:30:40 +02:00
Ari Koivula 5513744d24 Merge branch 'slices' 2017-01-31 16:14:30 +02:00
Ari Koivula 7868155c06 Add tests for --slices 2017-01-31 15:58:00 +02:00
Ari Koivula 52904d3e9f Add --slices=tiles and --slices=wpp
This encapsulates tiles or WPP rows into their own slices, making
it possible to send them as soon as they are done, instead of waiting
for the other substreams to finish and coding the substream offsets
in the slice header.
2017-01-31 15:44:23 +02:00
Ari Koivula 0d4d0e869c Add support for independent slices
Not used yet, but they work.
2017-01-31 15:11:50 +02:00
Ari Koivula 46ae382498 Fix bugs with slice header
These fixes allow more than one slice to be used to code a picture.
- Use correct number of bits to code the slice segment address.
- Don't offset_len_minus1 for slices without substreams.
2017-01-31 14:01:59 +02:00
Ari Koivula f1fc0de2bf Write slice headers to the parent stream
Appending to the child stream doesn't work is the child is a leaf
slice state.

Simplifies flow by removing distinction between tile and slice. Now
that slice headers are written in the parent stream, there is zero
difference between tiles and slices from bitstream point of view.
2017-01-31 13:55:05 +02:00
Ari Koivula 04cd875b2c Move substream finalization to LCU coding job
Having some of the termination bits in the LCU coding and some in the
substream finalization was needlessly confusing. Doing substream
finalization directly after LCU coding makes it easy to verify that the
finalization is done correctly.

Removes one job per WPP row from the job queue.

Removes kvz_cabac_flush, because I don't like bits being put into the
bitstream implicitly. Better to have it all in the open.
2017-01-31 13:01:57 +02:00
Ari Koivula ead490b7b7 Write a new slice NAL for every slice 2017-01-31 12:36:18 +02:00
Ari Koivula cd496bf50b Move first_nal_in_au to encoder_state->frame
Needed for writing NALs from encoder_state_write_bitstream_children
2017-01-31 12:28:28 +02:00
Arttu Ylä-Outinen 1e6463c08b Fix inter bipred search
When the number of merge candidates was five, biprediction search would
read past the bounds of the priority list arrays. Fixed to limit the
search to the first four candidates.
2017-01-31 18:23:12 +09:00
Ari Lemmetti 2c069a3e5f Prevent unnecessary cu search
Prevent further analysis as soon as it is known that splitting can not improve cost
2017-01-30 16:21:41 +02:00
Arttu Ylä-Outinen 0ee6321a6b Merge branch 'fixes' 2017-01-29 19:00:38 +09:00
Arttu Ylä-Outinen 9b889c3fab Fix reading ROI files
- Checks the return value of fopen when opening the ROI file. Fixes
  a segfault when the file cannot be opened.
- Check that the width and height are positive. Fixes reading past the
  end of the delta QP array in kvz_set_lcu_lambda_and_qp.
- Check for overflow in width * height. Fixes an overflow resulting in
  a segfault.
- Properly check that fscanf succeeds. Fixes silently accepting ROI
  files that are too short.
- Properly close the FILE pointer.
2017-01-29 18:57:27 +09:00
Arttu Ylä-Outinen 46c9a483c3 Fix inter search for small SMP and AMP blocks
The function search_pu_inter_ref incorrectly rounded the coordinates of
the block to down to a multiple 8 pixels. Small SMP and AMP blocks may
start at coordinates that are not multiples of 8. Fixed by removing the
rounding.

Fixes a failing assert when --mv-constraint is used with --smp or --amp.
2017-01-29 13:34:50 +09:00
Arttu Ylä-Outinen fb10b56b82 Fix checking if a low delay GOP structure is used
Stops assuming that having cfg->gop_lowdelay set means that GOP
structure is used since it is possible that cfg->gop_lowdelay is true
but cfg->gop_len is zero. Adds checks for cfg->gop_len where needed.

Fixes a possible division by zero in kvz_encoder_feed_frame.
2017-01-28 21:56:00 +09:00
Arttu Ylä-Outinen 4f56b04239 Drop an unnecessary conditional
Drop a conditional for depth > MAX_DEPTH in search_cu. The depth cannot
be greater than MAX_DEPTH (== 3) since an earlier if-clause checks that
it is less than MAX_PU_DEPTH (== 4).
2017-01-28 21:35:27 +09:00
Ari Koivula 937a764987 Fix bug in --mv-constraint
Subpixel motion estimation return 0-vector when no subpixel vector is
within the constraint. Fix is to not call subpixel motion estimation
when the integer vector is not within the constraint.
2017-01-26 09:55:57 +02:00
Ari Koivula c1cc46d1ed Add CI tests for --mv-constraint 2017-01-26 09:43:08 +02:00
Ari Koivula 4a0121ac42 Add --roi parameter
Adds region of interest coding capability.

Works by reading a file of delta QP values which will then be applied
to each frame at LCU level.
2017-01-26 09:14:14 +02:00
Ari Koivula 7ec5f78792 Merge branch 'refac' 2017-01-20 00:14:55 +02:00
Ari Koivula 6f61836989 Refactor kvz_rdoq_sign_hiding
Rename and reorder everything to make more sense.

- Moved input tables into their own struct and renamed them to what
  they actually represent.
- Renamed pretty much every variable to comform to our style and
  to make sense.
- Removed the lastCG stuff, as the function already gets passed the
  last coeff anyway. (it was named width, what the hell?)
2017-01-19 23:58:17 +02:00
Ari Koivula a85390d0ac Clean up code using the fixed point frac bit tables
This is to prepare for changing the code using the floating point table
to use the fixed point table instead.

This also allows reducing the size of the fractional part, which was
useful for finding every place where the the fixed point presentation
is relied upon.
2017-01-19 20:20:51 +02:00
Ari Koivula 24a69c7467 Refactor luma deblocking
Changes luma deblocking to use gather and scatter instead of reading
to and writing from here and there in memory. Should make them
faster and easier to vectorize, or at least cleaner.

Splits strong and weak luma deblocking to two functions, as they have
almost nothing in common.
2017-01-17 22:13:39 +02:00
Ari Koivula 4cb2fca924 Refactor deblock decision 2017-01-17 19:34:17 +02:00
Arttu Ylä-Outinen 2318bd77ed Merge branch 'rate-control-improvements' 2017-01-11 15:54:42 +09:00
Arttu Ylä-Outinen 05794c3548 Add missing static to function lambda_to_qp 2017-01-11 15:53:55 +09:00
Arttu Ylä-Outinen ee518e8ac4 Take header bits into account in rate control 2017-01-11 15:53:55 +09:00
Arttu Ylä-Outinen c219d3cd94 Fix deblock when CU QP delta is enabled
Fixes deblock functions so that they use the correct QP for the filtered
edge. Adds field qp to cu_info_t.
2017-01-11 15:53:22 +09:00
Arttu Ylä-Outinen 82a98180e4 Clip LCU lambda to reduce quality fluctuation
Limits lambdas for each LCU based on the computed lambda from the
previous frame and the frame-level lambda.
2017-01-09 01:24:23 +09:00
Arttu Ylä-Outinen 93172fd251 Use separate alpha, beta and lambda for each LCU
Changes rate control to use the alpha and beta values stored in
lcu_stats_t instead of the frame-level values when selecting lambda and
QP for an LCU.
2017-01-09 01:24:23 +09:00
Arttu Ylä-Outinen 3af4e9cc8a Allocate bits separately for each LCU
Bits are allocated based on the costs of the LCUs in the previous
completely coded frame.

Breaks deblock when rate control is used.
2017-01-09 01:24:23 +09:00
Arttu Ylä-Outinen ff5e5ec6d4 Record info about coded LCUs
Adds field lcu_stats to encoder_state_config_frame_t. The following data
is recorded for each LCU:
    - number of bits
    - squared cost
    - used lambda value
    - alpha parameter used for rate control
    - beta parameter used for rate control
2017-01-09 01:24:23 +09:00
Arttu Ylä-Outinen 2a4243acbe Refactor rate control
Moves all code related to setting QP and lambda values to rate_control
module.
2017-01-09 01:24:23 +09:00
Arttu Ylä-Outinen 71633889ce Enable CU QP delta when using rate control
When rate control is enabled, enable cu_qp_delta_enabled_flag in PPS
with diff_cu_qp_delta_depth set to 0. Also adds code for writing the QP
deltas and a new cabac context.
2017-01-09 01:24:23 +09:00
Arttu Ylä-Outinen 640ff94ecd Use separate lambda and QP for each LCU
Adds fields lambda, lambda_sqrt and qp to encoder_state_t. Drops field
cur_lambda_cost_sqrt from encoder_state_config_frame_t and renames
cur_lambda_cost to lambda.
2017-01-09 01:24:23 +09:00