Commit graph

310 commits

Author SHA1 Message Date
Ari Koivula 3b04d39db4 Take cabac bits into account on transform tree. 2014-10-08 10:31:54 +03:00
Ari Koivula 296f142d9e Retain coded block flag data during transform split search. 2014-10-08 10:31:54 +03:00
Ari Koivula 85dea10f3f Clean up transform split search.
- Remove unnecessary checks and comment.
2014-10-08 10:31:54 +03:00
Ari Koivula e1b801eb6f Add transform tree chroma cbf bits. 2014-10-08 10:31:23 +03:00
Ari Koivula 3868cc7ff1 Fix crash on inter search when --tr-depth-intra is used.
- Transform splits meant for intra modes were used for inter when inter mode
  was chosen, which caused an assert to be triggered if the split transform
  block didn't have any coefficients.
2014-10-03 19:29:06 +03:00
Ari Koivula 8a80845b91 Add chroma to transform split search. 2014-10-03 11:36:57 +03:00
Ari Koivula 51662e1081 Fix differences between cu_rd_cost_luma and rdo_cost_intra. 2014-10-03 11:36:57 +03:00
Ari Koivula bc7d7d5cb6 Add cu_info* as parameter to reconstruction functions.
- This is required so these functions can be used for searching. When NULL
  is given they take the CU from LCU struct as they did previously.

Conflicts:
	src/search.c
2014-10-03 11:36:56 +03:00
Ari Koivula ccc575e2c6 Disable transform tree bits. 2014-10-03 11:36:56 +03:00
Ari Koivula a0ab469c89 Disable rdo_cost_intra. 2014-10-03 11:36:56 +03:00
Ari Koivula c164978e21 Add FULL_CU_SPLIT_SEARCH macro for disabling cu split optimization. 2014-10-03 11:36:56 +03:00
Ari Koivula 549ac96438 Change costs to doubles to avoid rounding intermediate results.
- Helps with debugging.
2014-10-03 11:36:56 +03:00
Ari Koivula e591e89ade Add prediction mode to chroma reconstruction parameters.
- Just like in luma.
2014-10-03 11:36:56 +03:00
Ari Koivula bfa399c8fc Fix compiler warnings.
- Non-parenthesized parameter in a macro.
- Unused variables.
- Wrong const qualifiers.
- Signed/unsigned comparison.
2014-09-22 10:04:57 +03:00
Ari Koivula 0db7d8d20f test cu split cost 2014-09-15 15:42:03 +03:00
Ari Koivula 3c73892609 Fix transform split search.
- Redo the search with the best mode to make sure the tr_depth parameters are
  correct.
2014-09-11 10:56:53 +03:00
Ari Koivula 46b6b1243b Add --rd=3 mode and enable searching of intra depth 0.
- intra_build_reference_border was overflowing at depth 0 because it uses
  arrays just large enough to accommodate 32x32 transforms, which is the
  biggest transform.
- For similar reasons search_intra_rough doesn't work at depth 0.
- The --rd=3 mode tries all modes with transform search. It also works without
  rough search so it was used to test depth 0 search. If --rd=3 is not on intra
  split at depth 0 is not searched for.

Conflicts:
	src/search.c
2014-09-11 10:54:41 +03:00
Ari Koivula c5fa824347 Rebase transform split search. 2014-09-08 14:13:59 +03:00
Laurent Fasnacht f3c311fe1a Fix commit 8502f3d 2014-08-11 15:17:15 +02:00
Laurent Fasnacht f9bffe35a5 Log tile id in sad perf log 2014-08-11 11:57:08 +02:00
Laurent Fasnacht 6a937de9b2 Fix search_cu log 2014-08-11 11:57:08 +02:00
Laurent Fasnacht 8502f3d850 Improve logging 2014-08-11 11:57:07 +02:00
Ari Koivula f605d6c35b Align intra buffers to 32 bytes for 256 bit SIMD instructions. 2014-07-11 17:26:19 +03:00
Laurent Fasnacht 72abc69b3d Measure time for SAD in _DEBUG mode 2014-07-08 11:42:58 +02:00
Laurent Fasnacht e64a692780 Add CU type in threadqueue.log 2014-07-08 09:06:31 +02:00
Laurent Fasnacht 946e3b9651 Log search_cu to threadqueue.log 2014-07-07 10:50:05 +02:00
Ari Koivula 7ecf78bb70 Use sqrt lambda cost for searches not using SSD.
- Add encoder_state->global->cur_lambda_cost_sqrt.

- Use sqrt lambda for inter search and rough intra search.

- The effect on inter is around 10-20% bdrate. The effect on intra is smaller
  and non-existent when --rd=2 is enabled, as the intra search refinement was
  already done with SSD and correct lambda.
2014-06-26 13:56:38 +03:00
Ari Koivula 60ecc6baae Remove unused stuff. 2014-06-17 16:20:01 +03:00
Ari Koivula 94bc457b6c Add option to disable fast intra search. 2014-06-17 15:32:05 +03:00
Ari Koivula e27fc875c0 Clean up intra search. 2014-06-17 15:09:12 +03:00
Ari Koivula e4d70ac1ab Use more starting points for smaller blocks in intra search. 2014-06-17 13:28:27 +03:00
Ari Koivula 9911c7553b Avoid unnecessary intra dir searching. 2014-06-17 13:11:35 +03:00
Ari Koivula bd16a55b9b Always check DC and planar intra modes.
- At least one of them is always in predicted modes, but to make sure they
  are both included add them explicitly.
2014-06-17 12:51:15 +03:00
Ari Koivula 70740da123 Add smarter rough intra search.
- Directional intra mode search is done using halving search from the best
  known mode. Starting modes are vertical, horizontal and the 3 diagonal
  modes.

Conflicts:
	src/search.c
2014-06-17 12:33:10 +03:00
Marko Viitanen 0e2fe9e7ff Changed intra search to skip some modes speeding it up 2014-06-17 12:32:29 +03:00
Marko Viitanen a1c3cfe944 Moved intra mode cost calculation to a function
Conflicts:
	src/search.c
2014-06-17 12:32:29 +03:00
Marko Viitanen eb7d46f9ef Modify CU split cost. 2014-06-17 12:30:32 +03:00
Ari Koivula b3c15b8f94 Merge branch 'owf' 2014-06-16 16:07:41 +03:00
Laurent Fasnacht 91de92134f Constrain the search not to go under the LCU below if OWF is enabled 2014-06-16 14:27:56 +02:00
Ari Koivula 1c97a10a6d Move intra SAD and SATD functions under strategies. 2014-06-16 12:13:41 +03:00
Laurent Fasnacht f4187dd10c cu_array data structure 2014-06-16 10:54:57 +02:00
Ari Koivula 87ca828a63 Correct intra sad function labels.
- These haven't been 16 bit for a long time.
2014-06-16 10:45:10 +03:00
Laurent Fasnacht 6b408b5904 No-copy works with --no-sao --no-deblock 2014-06-12 11:47:30 +02:00
Laurent Fasnacht 27a49d287d Big refactor to use videoframe, image_list, and image instead of picture* 2014-06-10 09:19:06 +02:00
Laurent Fasnacht 2456c65822 Replace accesses to picture->cu_array with picture_get_cu and picture_get_cu_const 2014-06-05 10:41:58 +02:00
Ari Koivula e585da37e5 Give correct transform depth to RDOQ.
Conflicts:
	src/search.c
2014-05-28 15:47:49 +03:00
Ari Koivula ddc02cc09e Avoid regenerating reference pixels for every rdo mode. 2014-05-22 13:18:28 +03:00
Ari Koivula dbe13d0cba Separate sad intra search from rdo search. 2014-05-22 12:47:45 +03:00
Ari Koivula 19ce21e07c Split final cost to luma and chroma functions. 2014-05-22 09:45:00 +03:00
Ari Koivula a6962e2974 Separate intra transform coding to luma and chroma functions. 2014-05-22 09:40:34 +03:00
Ari Koivula f9a603e4ea Move intra mode search form intra module to search module.
- Make the actual intra prediction function global.

- Move the rdo stuff to rdo module.
2014-05-19 16:12:02 +03:00
Ari Koivula 656b0a321b Add chroma mode to lcu_set_intra_mode.
- This is needed for intra split.
2014-05-19 15:58:54 +03:00
Ari Koivula 921f58b249 Add tr_split to lcu_set_intra_mode. 2014-05-19 15:58:54 +03:00
Ari Koivula 846b608125 Add transform split recursion to intra reconstruction. 2014-05-19 15:58:54 +03:00
Ari Koivula c9a8756fbd Fix NxN scan mode for lcu_get_final_cost.
- Scan mode was always selected according to the first PU mode.
2014-05-15 16:20:35 +03:00
Marko Viitanen b08047cce9 Fixed intra chroma mode selection 2014-05-15 09:50:05 +03:00
Tapio Katajisto 4d879945b2 Fixed cost calculations in fme 2014-05-15 03:42:42 +00:00
Ari Koivula f0e990905e Remove chroma mode "36".
- It's an unnecessary chore to handle this special case everywhere (it means
  chroma_mode == intra_mode). Better just to use the actual mode.
2014-05-14 19:56:35 +03:00
Laurent Fasnacht bb86f24000 Take advantage of the new buffers to remove uneeded item assignment 2014-05-14 13:27:13 +02:00
Laurent Fasnacht 6607c9f563 Use new buffers for search 2014-05-14 13:27:12 +02:00
Laurent Fasnacht c257c4b863 Add const for the buffers 2014-05-14 13:27:12 +02:00
Ari Koivula 9147b7acbf Split residual quantization to separate luma and chroma function. 2014-05-14 11:19:48 +03:00
Tapio Katajisto cc92cfee18 Added few warnings to Makefile
Cleaned fme code a bit
2014-05-14 01:49:34 +00:00
Tapio Katajisto efc43c8b3a Added fractional pixel motion estimation
Added farctional mv support for inter recon

Added 1/8-pel chroma and 1/4-pel luma interpolation
2014-05-14 01:42:02 +00:00
Ari Koivula 1c38209cab Add missing include. 2014-05-13 09:33:05 +03:00
Laurent Fasnacht 0e6f1c99fc Refactor picture to remove hidden dependency between slice and tiles
picture.type -> encoder_state->global->pictype
picture.slicetype -> encoder_state->global->slicetype
picture.slice_sao_luma_flag -> 1 (was constant)
picture.slice_sao_chroma_flag -> 1 (was constant)

This may be changed later. For now it's better to avoid having slice related stuff in picture.
2014-05-07 11:55:48 +02:00
Laurent Fasnacht 6c6adf18c7 Refactor encoder_state 2014-05-07 11:47:31 +02:00
Ari Koivula bdc16d2612 Improve cu_info coded block flag data structure a bit.
- It works just like the old structure except that the flags are checked with
  bitmasks instead of having the flag value be propagated upwards. There isn't
  really any benefit to this because the flags still have to be propagated to
  parent CUs.

- Wrapped them inside a struct to make copying them easier. (Just need to copy
  the struct instead of making individual copies)
2014-05-06 18:28:04 +03:00
Ari Koivula 4490e8afd6 Remove depth dimension from picture->cu_array.
- It isn't used for anything anymore.

- It was used in the past to hold information during search, but now that
  information is held in lcu_t structs.
2014-04-28 10:18:22 +03:00
Laurent Fasnacht 5e7945888a Inter-frame prediction with tiles works.
Many thanks to Jean-Hugues Recolin for the insightful comments about shifts!
2014-04-25 09:28:00 +02:00
Laurent Fasnacht 571a373f69 Use tile offset in search 2014-04-24 08:38:24 +02:00
Laurent Fasnacht 19b1642aa2 Removed all cabac parameters (cabac is part of encoder_state) 2014-04-22 11:46:53 +02:00
Laurent Fasnacht 5fea5875a5 Huge refactoring
Split some parts of encoder_control into encoder_state
(idea: encoder_control is immutable)

Goal is to allow multiple substreams in the future.
2014-04-22 10:39:12 +02:00
Laurent Fasnacht 3396264f3c Moved g_cur_lambda_cost into encoder_control.cur_lambda_cost 2014-04-17 12:00:21 +02:00
Laurent Fasnacht 534013be77 Remove g_lambda_cost 2014-04-17 11:49:27 +02:00
Laurent Fasnacht 4a9c239027 Remove g_bitdepth 2014-04-17 11:13:13 +02:00
Laurent Fasnacht 7a2b883059 Remove encoder_input width, height, height_in_lcu, and width_in_lcu 2014-04-17 11:13:12 +02:00
Laurent Fasnacht 9ac3b7bf2b encoder->in.cur_pic --> cur_pic 2014-04-17 11:13:10 +02:00
Laurent Fasnacht 52ae027b3a Avoid undefined behavior in memcpy calls
"The memcpy() function shall copy n bytes from the object pointed to by s2 into the object pointed to by s1. If copying takes place between objects that overlap, the behavior is undefined."
2014-04-15 06:30:21 +02:00
Laurent Fasnacht 78c579053a encoder_control should be const in nearly all the code 2014-04-14 10:56:06 +02:00
Ari Koivula 5fa5e01e05 Merge branch 'intra-cleanup'
Conflicts:
	src/intra.c
	src/intra.h
	src/search.c
2014-04-10 13:51:14 +03:00
Ari Koivula 088dd9ab96 Clean up intra mode search.
- This changes the bitstream a little bit, because it changes the order in
  which the modes are tried and when two modes have the same cost the first
  one is chosen.

- Dst buffer was removes as it was no longer used.
2014-04-10 10:25:57 +03:00
Marko Viitanen 43ae0a3b9a Implemented RDO cost calculation to Intra modes 2014-04-10 10:25:20 +03:00
Marko Viitanen c38ec1aa10 Added commandline option for RDO (--rd) 2014-04-09 12:29:15 +03:00
Ari Koivula 92ac5025f9 Take intra mode based coeff scan mode into account for coeff bit cost.
- Previously only diagonal scan mode, the most common one, would be used.

- This improved bdrate by 0.1-0.5 % for p0 and 0-0.2 % for p60.
2014-04-09 10:44:44 +03:00
Ari Koivula 0251bf5a1a Improve calculation of chroma coding cost for 4x4 blocks.
- Adds calculation of chroma coefficient cost for 4x4 blocks.

- Previously there was no cost. Now the cost is added to the first prediction
  block for NxN.

- This fix should improve bdrate by about 1%.
2014-04-08 12:43:26 +03:00
Ari Koivula 3c0977c7f3 Fix buffer overflow on copying of reference pixels.
- Valgrind noticed this.

- Shouldn't affect anything as the buffer overflowed to pixel buffers which
  were initialized later.
2014-04-04 17:28:56 +03:00
Marko Viitanen e15a86268d Clean up tabs and whitespaces 2014-04-04 16:04:44 +03:00
Laurent Fasnacht 8a14bd3b7b Remove cabac global variable 2014-04-04 14:26:40 +02:00
Marko Viitanen c5ba5eb3c8 Use RDO in final_cost 2014-04-04 14:10:49 +03:00
Ari Koivula d44d1837bb Remove unreferenced parameters.
- Working towards issue #11.
2014-04-04 12:56:24 +03:00
Ari Koivula 46d33d3945 Resolve unsigned/signed mismatch warnings.
- Working towards issue #11.
2014-04-04 12:56:23 +03:00
Ari Koivula 313466fdff Remove unused variables.
- Working towards issue #11.

- Either removed or redefined variables to not cause a warning.
2014-04-03 15:37:59 +03:00
Marko Viitanen 0da8071300 Changed final cost (and transform skip) error function from SAD to SSD 2014-04-02 14:51:39 +03:00
Ari Koivula bbd1202f90 Replace old LCU reference pixels initialization with new ones.
In the future we might even have just a const pointer to reference pixels.
2014-03-19 16:01:31 +02:00
Ari Koivula 7328fc2897 Add special handling for bottom right LCU pixel.
I didn't take into account that the reference pixel on the top-left of the
LCU gets over written if we just replace the top reference pixels for
current LCU with the bottom reference pixels after doing the search.
To handle this I copy the pixel that gets overwritten to the vertical
reference pixels.
2014-03-19 16:01:31 +02:00
Ari Koivula 57ce7e990b Add new reference pixel buffer management to encoding loop.
- This is necessary because after we add in-loop filters to be done per LCU,
  the reconstruction buffer will have the deblocked pixels. We only need the
  edge-pixels for intra prediction though so we just save those.

- Right now it only copies the pixels and passes them on to search, where
  the copied pixels are asserted to be the same ones we copy from
  reconstruction buffer.

- New yuv_t struct added for arrays of dynamic length. We might want to change
  other buffers to use it or something like it in the future.
2014-03-19 16:01:30 +02:00
Ari Koivula 6a384112cc Remove old unnecessary functions from search module.
- As we move to per-LCU encoding only search_lcu will be necessary.

- Function search_best_mode is no longer used.
2014-03-17 18:04:41 +02:00
Marko Viitanen c7ee176480 Removed unused function search_best_mode() 2014-03-17 12:56:04 +02:00
Ari Koivula 34d4e74cd6 Fix selection of bitcost for NxN mode intra CUs. 2014-03-12 18:05:42 +02:00