Commit graph

2771 commits

Author SHA1 Message Date
Ari Koivula d893474bab Fix encoder getting stuck on OS-X
Main thread was stuck looping on pthread_cond_timedwait because
the abs time given on OS-X had already passed and the wait
returned immediately without releasing the mutex to allow worker
threads to proceed.

Fix was to use the gettimeofday, which returns real time instead
of monotonic, which is what pthread_cond_timedwait wants.
2017-02-02 17:27:46 +02:00
Ari Koivula 4ceda1908b Fix OS-X compiler warning
rdo.c:475:25: warning: absolute value function 'abs' given an argument of type 'int64_t' (aka 'long long') but has parameter
       of type 'int' which may cause truncation of value [-Wabsolute-value]
         current.cost = -abs(quant_cost_in_bits) + (bits << PRECISION_INC);
                         ^
rdo.c:475:25: note: use function 'llabs' instead
         current.cost = -abs(quant_cost_in_bits) + (bits << PRECISION_INC);
2017-02-01 18:09:17 +02:00
Ari Koivula c7d536bbcd Fix OS-X compiler warning
cfg.c:1024:74: warning: format specifies type 'size_t' (aka 'unsigned
long') but the argument has type 'unsigned long long'
       [-Wformat]
       fprintf(stderr, "Too large ROI size: %llu (maximum %zu).\n", size, SIZE_MAX);
2017-02-01 18:09:04 +02:00
Ari Koivula 4467506ef1 Add missing kvz_ prefix 2017-01-31 18:38:02 +02:00
Ari Koivula ed3bd898fd Remove Exp-Golomb lookup table
This table takes 256kB and isn't used very much. Au revoir!
2017-01-31 18:31:05 +02:00
Ari Koivula e85266efe7 Update docs 2017-01-31 16:30:40 +02:00
Ari Koivula 5513744d24 Merge branch 'slices' 2017-01-31 16:14:30 +02:00
Ari Koivula 7868155c06 Add tests for --slices 2017-01-31 15:58:00 +02:00
Ari Koivula 52904d3e9f Add --slices=tiles and --slices=wpp
This encapsulates tiles or WPP rows into their own slices, making
it possible to send them as soon as they are done, instead of waiting
for the other substreams to finish and coding the substream offsets
in the slice header.
2017-01-31 15:44:23 +02:00
Ari Koivula 0d4d0e869c Add support for independent slices
Not used yet, but they work.
2017-01-31 15:11:50 +02:00
Ari Koivula 46ae382498 Fix bugs with slice header
These fixes allow more than one slice to be used to code a picture.
- Use correct number of bits to code the slice segment address.
- Don't offset_len_minus1 for slices without substreams.
2017-01-31 14:01:59 +02:00
Ari Koivula f1fc0de2bf Write slice headers to the parent stream
Appending to the child stream doesn't work is the child is a leaf
slice state.

Simplifies flow by removing distinction between tile and slice. Now
that slice headers are written in the parent stream, there is zero
difference between tiles and slices from bitstream point of view.
2017-01-31 13:55:05 +02:00
Ari Koivula 04cd875b2c Move substream finalization to LCU coding job
Having some of the termination bits in the LCU coding and some in the
substream finalization was needlessly confusing. Doing substream
finalization directly after LCU coding makes it easy to verify that the
finalization is done correctly.

Removes one job per WPP row from the job queue.

Removes kvz_cabac_flush, because I don't like bits being put into the
bitstream implicitly. Better to have it all in the open.
2017-01-31 13:01:57 +02:00
Ari Koivula ead490b7b7 Write a new slice NAL for every slice 2017-01-31 12:36:18 +02:00
Ari Koivula cd496bf50b Move first_nal_in_au to encoder_state->frame
Needed for writing NALs from encoder_state_write_bitstream_children
2017-01-31 12:28:28 +02:00
Arttu Ylä-Outinen 1e6463c08b Fix inter bipred search
When the number of merge candidates was five, biprediction search would
read past the bounds of the priority list arrays. Fixed to limit the
search to the first four candidates.
2017-01-31 18:23:12 +09:00
Ari Lemmetti 2c069a3e5f Prevent unnecessary cu search
Prevent further analysis as soon as it is known that splitting can not improve cost
2017-01-30 16:21:41 +02:00
Arttu Ylä-Outinen 0ee6321a6b Merge branch 'fixes' 2017-01-29 19:00:38 +09:00
Arttu Ylä-Outinen 9b889c3fab Fix reading ROI files
- Checks the return value of fopen when opening the ROI file. Fixes
  a segfault when the file cannot be opened.
- Check that the width and height are positive. Fixes reading past the
  end of the delta QP array in kvz_set_lcu_lambda_and_qp.
- Check for overflow in width * height. Fixes an overflow resulting in
  a segfault.
- Properly check that fscanf succeeds. Fixes silently accepting ROI
  files that are too short.
- Properly close the FILE pointer.
2017-01-29 18:57:27 +09:00
Arttu Ylä-Outinen 46c9a483c3 Fix inter search for small SMP and AMP blocks
The function search_pu_inter_ref incorrectly rounded the coordinates of
the block to down to a multiple 8 pixels. Small SMP and AMP blocks may
start at coordinates that are not multiples of 8. Fixed by removing the
rounding.

Fixes a failing assert when --mv-constraint is used with --smp or --amp.
2017-01-29 13:34:50 +09:00
Arttu Ylä-Outinen fb10b56b82 Fix checking if a low delay GOP structure is used
Stops assuming that having cfg->gop_lowdelay set means that GOP
structure is used since it is possible that cfg->gop_lowdelay is true
but cfg->gop_len is zero. Adds checks for cfg->gop_len where needed.

Fixes a possible division by zero in kvz_encoder_feed_frame.
2017-01-28 21:56:00 +09:00
Arttu Ylä-Outinen 4f56b04239 Drop an unnecessary conditional
Drop a conditional for depth > MAX_DEPTH in search_cu. The depth cannot
be greater than MAX_DEPTH (== 3) since an earlier if-clause checks that
it is less than MAX_PU_DEPTH (== 4).
2017-01-28 21:35:27 +09:00
Ari Koivula 937a764987 Fix bug in --mv-constraint
Subpixel motion estimation return 0-vector when no subpixel vector is
within the constraint. Fix is to not call subpixel motion estimation
when the integer vector is not within the constraint.
2017-01-26 09:55:57 +02:00
Ari Koivula c1cc46d1ed Add CI tests for --mv-constraint 2017-01-26 09:43:08 +02:00
Ari Koivula 4a0121ac42 Add --roi parameter
Adds region of interest coding capability.

Works by reading a file of delta QP values which will then be applied
to each frame at LCU level.
2017-01-26 09:14:14 +02:00
Ari Koivula 7ec5f78792 Merge branch 'refac' 2017-01-20 00:14:55 +02:00
Ari Koivula 6f61836989 Refactor kvz_rdoq_sign_hiding
Rename and reorder everything to make more sense.

- Moved input tables into their own struct and renamed them to what
  they actually represent.
- Renamed pretty much every variable to comform to our style and
  to make sense.
- Removed the lastCG stuff, as the function already gets passed the
  last coeff anyway. (it was named width, what the hell?)
2017-01-19 23:58:17 +02:00
Ari Koivula a85390d0ac Clean up code using the fixed point frac bit tables
This is to prepare for changing the code using the floating point table
to use the fixed point table instead.

This also allows reducing the size of the fractional part, which was
useful for finding every place where the the fixed point presentation
is relied upon.
2017-01-19 20:20:51 +02:00
Ari Koivula 24a69c7467 Refactor luma deblocking
Changes luma deblocking to use gather and scatter instead of reading
to and writing from here and there in memory. Should make them
faster and easier to vectorize, or at least cleaner.

Splits strong and weak luma deblocking to two functions, as they have
almost nothing in common.
2017-01-17 22:13:39 +02:00
Ari Koivula 4cb2fca924 Refactor deblock decision 2017-01-17 19:34:17 +02:00
Arttu Ylä-Outinen 2318bd77ed Merge branch 'rate-control-improvements' 2017-01-11 15:54:42 +09:00
Arttu Ylä-Outinen 05794c3548 Add missing static to function lambda_to_qp 2017-01-11 15:53:55 +09:00
Arttu Ylä-Outinen ee518e8ac4 Take header bits into account in rate control 2017-01-11 15:53:55 +09:00
Arttu Ylä-Outinen c219d3cd94 Fix deblock when CU QP delta is enabled
Fixes deblock functions so that they use the correct QP for the filtered
edge. Adds field qp to cu_info_t.
2017-01-11 15:53:22 +09:00
Arttu Ylä-Outinen 82a98180e4 Clip LCU lambda to reduce quality fluctuation
Limits lambdas for each LCU based on the computed lambda from the
previous frame and the frame-level lambda.
2017-01-09 01:24:23 +09:00
Arttu Ylä-Outinen 93172fd251 Use separate alpha, beta and lambda for each LCU
Changes rate control to use the alpha and beta values stored in
lcu_stats_t instead of the frame-level values when selecting lambda and
QP for an LCU.
2017-01-09 01:24:23 +09:00
Arttu Ylä-Outinen 3af4e9cc8a Allocate bits separately for each LCU
Bits are allocated based on the costs of the LCUs in the previous
completely coded frame.

Breaks deblock when rate control is used.
2017-01-09 01:24:23 +09:00
Arttu Ylä-Outinen ff5e5ec6d4 Record info about coded LCUs
Adds field lcu_stats to encoder_state_config_frame_t. The following data
is recorded for each LCU:
    - number of bits
    - squared cost
    - used lambda value
    - alpha parameter used for rate control
    - beta parameter used for rate control
2017-01-09 01:24:23 +09:00
Arttu Ylä-Outinen 2a4243acbe Refactor rate control
Moves all code related to setting QP and lambda values to rate_control
module.
2017-01-09 01:24:23 +09:00
Arttu Ylä-Outinen 71633889ce Enable CU QP delta when using rate control
When rate control is enabled, enable cu_qp_delta_enabled_flag in PPS
with diff_cu_qp_delta_depth set to 0. Also adds code for writing the QP
deltas and a new cabac context.
2017-01-09 01:24:23 +09:00
Arttu Ylä-Outinen 640ff94ecd Use separate lambda and QP for each LCU
Adds fields lambda, lambda_sqrt and qp to encoder_state_t. Drops field
cur_lambda_cost_sqrt from encoder_state_config_frame_t and renames
cur_lambda_cost to lambda.
2017-01-09 01:24:23 +09:00
Arttu Ylä-Outinen 435c387357 Refactor rate control
- Defines MIN_LAMBDA and MAX_LAMBDA constants.
- Moves resetting state->frame->cur_gop_bits_coded to rate_control.c.
- Changes gop_allocate_bits to return the number of bits allocated like
  pic_allocate_bits does.
2017-01-09 01:24:23 +09:00
Arttu Ylä-Outinen 6c4f2d196a Move fields from encoder_state_t to frame
Moves fields prepared and frame_done from encoder_state_t to
encoder_state_config_frame_t.
2017-01-09 01:24:23 +09:00
Arttu Ylä-Outinen 97863cdaa2 Fail encoder init when CQM file cannot be opened 2017-01-08 19:17:43 +09:00
Arttu Ylä-Outinen db5e750c7f Fix --threads=auto
When --threads=auto was given on the command line, cfg->threads was
actually set to zero, disabling threads altogether. Fixed to set
cfg->threads to -1, so that the number of threads is chosen
automatically.
2017-01-08 17:58:22 +09:00
Ari Koivula a9e45efcfc Add a fast lane for byte-aligned bitstream writes
The CABAC engine only writes to the bitstream when it has a full byte.
These writes are also always byte-aligned, so there is no need to even
check for stream alignment.

Speedup was around 3% with ultrafast and low QP.
2016-12-23 17:01:44 +02:00
Jaakko Laitinen deb63f735f Fix gop disabling 2016-12-20 14:25:13 +02:00
Ari Lemmetti 70a52f0e48 10-bit: add missing bit depth adjustment to ssd 2016-11-17 19:28:04 +02:00
Ari Koivula fa078102f1 Fix 32bit compilation
Got a warning about implicit cast from uint64_t to void*.
2016-11-17 17:53:57 +02:00
Ari Koivula 5ceec06bd3 Merge pull request #148 from Venti-/crypto
Crypto
2016-11-16 21:33:55 +02:00