Commit graph

2956 commits

Author SHA1 Message Date
Arttu Ylä-Outinen 1e6463c08b Fix inter bipred search
When the number of merge candidates was five, biprediction search would
read past the bounds of the priority list arrays. Fixed to limit the
search to the first four candidates.
2017-01-31 18:23:12 +09:00
Ari Lemmetti 2c069a3e5f Prevent unnecessary cu search
Prevent further analysis as soon as it is known that splitting can not improve cost
2017-01-30 16:21:41 +02:00
Arttu Ylä-Outinen 0ee6321a6b Merge branch 'fixes' 2017-01-29 19:00:38 +09:00
Arttu Ylä-Outinen 9b889c3fab Fix reading ROI files
- Checks the return value of fopen when opening the ROI file. Fixes
  a segfault when the file cannot be opened.
- Check that the width and height are positive. Fixes reading past the
  end of the delta QP array in kvz_set_lcu_lambda_and_qp.
- Check for overflow in width * height. Fixes an overflow resulting in
  a segfault.
- Properly check that fscanf succeeds. Fixes silently accepting ROI
  files that are too short.
- Properly close the FILE pointer.
2017-01-29 18:57:27 +09:00
Arttu Ylä-Outinen 46c9a483c3 Fix inter search for small SMP and AMP blocks
The function search_pu_inter_ref incorrectly rounded the coordinates of
the block to down to a multiple 8 pixels. Small SMP and AMP blocks may
start at coordinates that are not multiples of 8. Fixed by removing the
rounding.

Fixes a failing assert when --mv-constraint is used with --smp or --amp.
2017-01-29 13:34:50 +09:00
Arttu Ylä-Outinen fb10b56b82 Fix checking if a low delay GOP structure is used
Stops assuming that having cfg->gop_lowdelay set means that GOP
structure is used since it is possible that cfg->gop_lowdelay is true
but cfg->gop_len is zero. Adds checks for cfg->gop_len where needed.

Fixes a possible division by zero in kvz_encoder_feed_frame.
2017-01-28 21:56:00 +09:00
Arttu Ylä-Outinen 4f56b04239 Drop an unnecessary conditional
Drop a conditional for depth > MAX_DEPTH in search_cu. The depth cannot
be greater than MAX_DEPTH (== 3) since an earlier if-clause checks that
it is less than MAX_PU_DEPTH (== 4).
2017-01-28 21:35:27 +09:00
Ari Koivula 937a764987 Fix bug in --mv-constraint
Subpixel motion estimation return 0-vector when no subpixel vector is
within the constraint. Fix is to not call subpixel motion estimation
when the integer vector is not within the constraint.
2017-01-26 09:55:57 +02:00
Ari Koivula c1cc46d1ed Add CI tests for --mv-constraint 2017-01-26 09:43:08 +02:00
Ari Koivula 4a0121ac42 Add --roi parameter
Adds region of interest coding capability.

Works by reading a file of delta QP values which will then be applied
to each frame at LCU level.
2017-01-26 09:14:14 +02:00
Ari Koivula 7ec5f78792 Merge branch 'refac' 2017-01-20 00:14:55 +02:00
Ari Koivula 6f61836989 Refactor kvz_rdoq_sign_hiding
Rename and reorder everything to make more sense.

- Moved input tables into their own struct and renamed them to what
  they actually represent.
- Renamed pretty much every variable to comform to our style and
  to make sense.
- Removed the lastCG stuff, as the function already gets passed the
  last coeff anyway. (it was named width, what the hell?)
2017-01-19 23:58:17 +02:00
Ari Koivula a85390d0ac Clean up code using the fixed point frac bit tables
This is to prepare for changing the code using the floating point table
to use the fixed point table instead.

This also allows reducing the size of the fractional part, which was
useful for finding every place where the the fixed point presentation
is relied upon.
2017-01-19 20:20:51 +02:00
Ari Koivula 24a69c7467 Refactor luma deblocking
Changes luma deblocking to use gather and scatter instead of reading
to and writing from here and there in memory. Should make them
faster and easier to vectorize, or at least cleaner.

Splits strong and weak luma deblocking to two functions, as they have
almost nothing in common.
2017-01-17 22:13:39 +02:00
Ari Koivula 4cb2fca924 Refactor deblock decision 2017-01-17 19:34:17 +02:00
Arttu Ylä-Outinen 2318bd77ed Merge branch 'rate-control-improvements' 2017-01-11 15:54:42 +09:00
Arttu Ylä-Outinen 05794c3548 Add missing static to function lambda_to_qp 2017-01-11 15:53:55 +09:00
Arttu Ylä-Outinen ee518e8ac4 Take header bits into account in rate control 2017-01-11 15:53:55 +09:00
Arttu Ylä-Outinen c219d3cd94 Fix deblock when CU QP delta is enabled
Fixes deblock functions so that they use the correct QP for the filtered
edge. Adds field qp to cu_info_t.
2017-01-11 15:53:22 +09:00
Arttu Ylä-Outinen 82a98180e4 Clip LCU lambda to reduce quality fluctuation
Limits lambdas for each LCU based on the computed lambda from the
previous frame and the frame-level lambda.
2017-01-09 01:24:23 +09:00
Arttu Ylä-Outinen 93172fd251 Use separate alpha, beta and lambda for each LCU
Changes rate control to use the alpha and beta values stored in
lcu_stats_t instead of the frame-level values when selecting lambda and
QP for an LCU.
2017-01-09 01:24:23 +09:00
Arttu Ylä-Outinen 3af4e9cc8a Allocate bits separately for each LCU
Bits are allocated based on the costs of the LCUs in the previous
completely coded frame.

Breaks deblock when rate control is used.
2017-01-09 01:24:23 +09:00
Arttu Ylä-Outinen ff5e5ec6d4 Record info about coded LCUs
Adds field lcu_stats to encoder_state_config_frame_t. The following data
is recorded for each LCU:
    - number of bits
    - squared cost
    - used lambda value
    - alpha parameter used for rate control
    - beta parameter used for rate control
2017-01-09 01:24:23 +09:00
Arttu Ylä-Outinen 2a4243acbe Refactor rate control
Moves all code related to setting QP and lambda values to rate_control
module.
2017-01-09 01:24:23 +09:00
Arttu Ylä-Outinen 71633889ce Enable CU QP delta when using rate control
When rate control is enabled, enable cu_qp_delta_enabled_flag in PPS
with diff_cu_qp_delta_depth set to 0. Also adds code for writing the QP
deltas and a new cabac context.
2017-01-09 01:24:23 +09:00
Arttu Ylä-Outinen 640ff94ecd Use separate lambda and QP for each LCU
Adds fields lambda, lambda_sqrt and qp to encoder_state_t. Drops field
cur_lambda_cost_sqrt from encoder_state_config_frame_t and renames
cur_lambda_cost to lambda.
2017-01-09 01:24:23 +09:00
Arttu Ylä-Outinen 435c387357 Refactor rate control
- Defines MIN_LAMBDA and MAX_LAMBDA constants.
- Moves resetting state->frame->cur_gop_bits_coded to rate_control.c.
- Changes gop_allocate_bits to return the number of bits allocated like
  pic_allocate_bits does.
2017-01-09 01:24:23 +09:00
Arttu Ylä-Outinen 6c4f2d196a Move fields from encoder_state_t to frame
Moves fields prepared and frame_done from encoder_state_t to
encoder_state_config_frame_t.
2017-01-09 01:24:23 +09:00
Arttu Ylä-Outinen 97863cdaa2 Fail encoder init when CQM file cannot be opened 2017-01-08 19:17:43 +09:00
Arttu Ylä-Outinen db5e750c7f Fix --threads=auto
When --threads=auto was given on the command line, cfg->threads was
actually set to zero, disabling threads altogether. Fixed to set
cfg->threads to -1, so that the number of threads is chosen
automatically.
2017-01-08 17:58:22 +09:00
Ari Koivula a9e45efcfc Add a fast lane for byte-aligned bitstream writes
The CABAC engine only writes to the bitstream when it has a full byte.
These writes are also always byte-aligned, so there is no need to even
check for stream alignment.

Speedup was around 3% with ultrafast and low QP.
2016-12-23 17:01:44 +02:00
Jaakko Laitinen deb63f735f Fix gop disabling 2016-12-20 14:25:13 +02:00
Ari Lemmetti 70a52f0e48 10-bit: add missing bit depth adjustment to ssd 2016-11-17 19:28:04 +02:00
Ari Koivula fa078102f1 Fix 32bit compilation
Got a warning about implicit cast from uint64_t to void*.
2016-11-17 17:53:57 +02:00
Ari Koivula 5ceec06bd3 Merge pull request #148 from Venti-/crypto
Crypto
2016-11-16 21:33:55 +02:00
Ari Lemmetti c31207ea7d Optimize intra reference building
-Add function with reduced logic for the most common case
2016-11-16 18:28:42 +02:00
Ari Lemmetti 02c9e3746c Add AppVeyor badge 2016-11-16 17:12:36 +02:00
Ari Koivula 24f2a23ef8 Remove unnecessary crypto state
The frame does not need it's own crypto state, since it always has at
least one sub tile.
2016-11-16 13:58:41 +02:00
Ari Koivula 8951e34fd2 Change crypto.h stubs to print instead of assert 2016-11-16 13:58:41 +02:00
Wassim Hamidouche ea82c38906 correct memory allocation 2016-11-16 12:35:28 +02:00
Wassim Hamidouche da3e2d1d07 resolve parallel encryption 2016-11-16 12:35:28 +02:00
Ari Koivula b8a618e666 Fix problems with >8 bit input
Enforce bit depth promised by --input-bitdepth to avoid crashes when
larger values are provided.

Do endianess byte swap for all bytes when the buffer gets extended
to multiple of 8 pixels, and not just the number of input pixels.

Don't swap bytes on a little-endian system.
2016-11-13 19:58:54 +02:00
Ari Koivula 2c005cda25 Fix bug with sub-pixel motion estimation in tiles
The width of the tile was being used to index the frame pixel buffer
instead of the width of the buffer.
2016-11-07 15:53:52 +02:00
Ari Koivula bb33cd3901 Update lp-gop syntax in README 2016-11-04 17:22:24 +02:00
Ari Koivula 78a28e0338 Reformat --help message
- Reduce indentation to 6 spaces
- Word wrap everything to under 80 characters
- Remove defaults from options covered by presets
- Add a dash in front of argument descriptions
- Add --(no-) to names of parameters that accept it and remove mention
  of enabling or disabling
- Add executable and scripts as a dependancy to make docs
2016-11-04 15:40:28 +02:00
Ari Koivula 98a0d54b70 Merge branch 'dts-fix' 2016-10-28 19:06:22 +03:00
Ari Koivula d18de19d8a Fix DTS and PTS not being passed on through lib API
Fixes "cur_dts is invalid" warning from FFmpeg.
2016-10-28 19:05:47 +03:00
Ari Koivula 0c41c2ebd6 Make CLI set PTS for each input picture
This value is not represented in the HEVC bitstream, which is why it
was not set previously. FFmpeg sets and needs it however, so make the
CLI set it as well to make sure we handle it correctly.
2016-10-28 19:03:03 +03:00
Ari Koivula c9cfe8d76b Merge branch 'help' 2016-10-27 03:32:22 +03:00
Ari Koivula c7da5e981b Update README and manpage 2016-10-27 03:29:53 +03:00