Commit graph

329 commits

Author SHA1 Message Date
Ari Koivula f6fead6221 Fix crash on inter frames.
- If the bitcost was 0 it would underflow for skip mode. The bitcost is now
  checked before decrementing.
2014-10-21 18:11:39 +03:00
Ari Koivula dfc67b766a Disable rd1 chroma search.
- The bdrate improvement isn't really worth the time it takes, so enable it
  only for rd3 untill it can be made faster or better.
2014-10-16 13:59:20 +03:00
Ari Koivula e9b8d9b889 Fix gcc warnings.
- Remove unused variables.
- Change intra prediction functions to take their inputs as const pointers.
- Change intra_get_pred to take two pointers instead of an array of pointers,
  because the warnings got just too exotic.
2014-10-16 13:17:46 +03:00
Ari Koivula 4bac52d9b6 Merge branch 'intra' 2014-10-16 13:11:23 +03:00
Ari Koivula 02ec26fcea Try different number of chroma intra modes for different depths.
- And avoid doing extra work if no extra modes are tested for certain depths.
2014-10-16 12:21:36 +03:00
Ari Koivula 3cf5e422e8 Make fast chroma mode search select modes for slower chroma search. 2014-10-16 12:21:36 +03:00
Ari Koivula d12dbd4aa0 Add fast intra chroma mode search. 2014-10-16 12:21:08 +03:00
Ari Koivula 3e6023dfb5 Rename search constants and set sane defaults. 2014-10-16 03:08:11 +03:00
Ari Koivula 8a407b0313 Estimate luma and chroma intra mode bits separately.
- Remove cu_info.intra[].cost and bitcost as unnecessary.
- Add luma_mode_bits to complement chroma_mode_bits and remove
  intra_pred_ratecost as unneccessary. Difference is that intra_pred_ratecost
  was more coarse and included chroma mode with the assumption that it would
  be the same as chroma.
2014-10-16 03:08:11 +03:00
Ari Koivula c9e212ba92 Add intra chroma mode search.
- Based on full chroma reconstruction so enabled only for --rd=2.
2014-10-16 03:07:50 +03:00
Ari Koivula a469c059a5 Take chroma tr-skip bits into account. 2014-10-13 10:48:39 +03:00
Ari Koivula 7a5cf5d865 Add trskip mode cost to fast trskip mode decision. 2014-10-13 10:45:41 +03:00
Ari Koivula f164a5ba79 Add fast transform skip estimation to rough intra search. 2014-10-13 10:42:24 +03:00
Ari Koivula 28d1532578 Make rd=1 use cabac for coeff cost estimation. 2014-10-08 12:50:03 +03:00
Ari Koivula cbb2aa75b7 Add macros for adjusting weight of distortion between luma and chroma.
- Everything needs to have a short name because windows has a maximum path
  length limitation that is breaking my testing framework.
2014-10-08 10:31:54 +03:00
Ari Koivula 49ad845c33 Add cabac bits for part_mode. 2014-10-08 10:31:54 +03:00
Ari Koivula b6710e7893 Add cabac bits for cu split flag. 2014-10-08 10:31:54 +03:00
Ari Koivula 38b224cf69 Change rest of cu split search costs to double. 2014-10-08 10:31:54 +03:00
Ari Koivula 17473624d3 Add transform tree bit costs for cbf_luma. 2014-10-08 10:31:54 +03:00
Ari Koivula 3b04d39db4 Take cabac bits into account on transform tree. 2014-10-08 10:31:54 +03:00
Ari Koivula 296f142d9e Retain coded block flag data during transform split search. 2014-10-08 10:31:54 +03:00
Ari Koivula 85dea10f3f Clean up transform split search.
- Remove unnecessary checks and comment.
2014-10-08 10:31:54 +03:00
Ari Koivula e1b801eb6f Add transform tree chroma cbf bits. 2014-10-08 10:31:23 +03:00
Ari Koivula 3868cc7ff1 Fix crash on inter search when --tr-depth-intra is used.
- Transform splits meant for intra modes were used for inter when inter mode
  was chosen, which caused an assert to be triggered if the split transform
  block didn't have any coefficients.
2014-10-03 19:29:06 +03:00
Ari Koivula 8a80845b91 Add chroma to transform split search. 2014-10-03 11:36:57 +03:00
Ari Koivula 51662e1081 Fix differences between cu_rd_cost_luma and rdo_cost_intra. 2014-10-03 11:36:57 +03:00
Ari Koivula bc7d7d5cb6 Add cu_info* as parameter to reconstruction functions.
- This is required so these functions can be used for searching. When NULL
  is given they take the CU from LCU struct as they did previously.

Conflicts:
	src/search.c
2014-10-03 11:36:56 +03:00
Ari Koivula ccc575e2c6 Disable transform tree bits. 2014-10-03 11:36:56 +03:00
Ari Koivula a0ab469c89 Disable rdo_cost_intra. 2014-10-03 11:36:56 +03:00
Ari Koivula c164978e21 Add FULL_CU_SPLIT_SEARCH macro for disabling cu split optimization. 2014-10-03 11:36:56 +03:00
Ari Koivula 549ac96438 Change costs to doubles to avoid rounding intermediate results.
- Helps with debugging.
2014-10-03 11:36:56 +03:00
Ari Koivula e591e89ade Add prediction mode to chroma reconstruction parameters.
- Just like in luma.
2014-10-03 11:36:56 +03:00
Ari Koivula bfa399c8fc Fix compiler warnings.
- Non-parenthesized parameter in a macro.
- Unused variables.
- Wrong const qualifiers.
- Signed/unsigned comparison.
2014-09-22 10:04:57 +03:00
Ari Koivula 0db7d8d20f test cu split cost 2014-09-15 15:42:03 +03:00
Ari Koivula 3c73892609 Fix transform split search.
- Redo the search with the best mode to make sure the tr_depth parameters are
  correct.
2014-09-11 10:56:53 +03:00
Ari Koivula 46b6b1243b Add --rd=3 mode and enable searching of intra depth 0.
- intra_build_reference_border was overflowing at depth 0 because it uses
  arrays just large enough to accommodate 32x32 transforms, which is the
  biggest transform.
- For similar reasons search_intra_rough doesn't work at depth 0.
- The --rd=3 mode tries all modes with transform search. It also works without
  rough search so it was used to test depth 0 search. If --rd=3 is not on intra
  split at depth 0 is not searched for.

Conflicts:
	src/search.c
2014-09-11 10:54:41 +03:00
Ari Koivula c5fa824347 Rebase transform split search. 2014-09-08 14:13:59 +03:00
Laurent Fasnacht f3c311fe1a Fix commit 8502f3d 2014-08-11 15:17:15 +02:00
Laurent Fasnacht f9bffe35a5 Log tile id in sad perf log 2014-08-11 11:57:08 +02:00
Laurent Fasnacht 6a937de9b2 Fix search_cu log 2014-08-11 11:57:08 +02:00
Laurent Fasnacht 8502f3d850 Improve logging 2014-08-11 11:57:07 +02:00
Ari Koivula f605d6c35b Align intra buffers to 32 bytes for 256 bit SIMD instructions. 2014-07-11 17:26:19 +03:00
Laurent Fasnacht 72abc69b3d Measure time for SAD in _DEBUG mode 2014-07-08 11:42:58 +02:00
Laurent Fasnacht e64a692780 Add CU type in threadqueue.log 2014-07-08 09:06:31 +02:00
Laurent Fasnacht 946e3b9651 Log search_cu to threadqueue.log 2014-07-07 10:50:05 +02:00
Ari Koivula 7ecf78bb70 Use sqrt lambda cost for searches not using SSD.
- Add encoder_state->global->cur_lambda_cost_sqrt.

- Use sqrt lambda for inter search and rough intra search.

- The effect on inter is around 10-20% bdrate. The effect on intra is smaller
  and non-existent when --rd=2 is enabled, as the intra search refinement was
  already done with SSD and correct lambda.
2014-06-26 13:56:38 +03:00
Ari Koivula 60ecc6baae Remove unused stuff. 2014-06-17 16:20:01 +03:00
Ari Koivula 94bc457b6c Add option to disable fast intra search. 2014-06-17 15:32:05 +03:00
Ari Koivula e27fc875c0 Clean up intra search. 2014-06-17 15:09:12 +03:00
Ari Koivula e4d70ac1ab Use more starting points for smaller blocks in intra search. 2014-06-17 13:28:27 +03:00
Ari Koivula 9911c7553b Avoid unnecessary intra dir searching. 2014-06-17 13:11:35 +03:00
Ari Koivula bd16a55b9b Always check DC and planar intra modes.
- At least one of them is always in predicted modes, but to make sure they
  are both included add them explicitly.
2014-06-17 12:51:15 +03:00
Ari Koivula 70740da123 Add smarter rough intra search.
- Directional intra mode search is done using halving search from the best
  known mode. Starting modes are vertical, horizontal and the 3 diagonal
  modes.

Conflicts:
	src/search.c
2014-06-17 12:33:10 +03:00
Marko Viitanen 0e2fe9e7ff Changed intra search to skip some modes speeding it up 2014-06-17 12:32:29 +03:00
Marko Viitanen a1c3cfe944 Moved intra mode cost calculation to a function
Conflicts:
	src/search.c
2014-06-17 12:32:29 +03:00
Marko Viitanen eb7d46f9ef Modify CU split cost. 2014-06-17 12:30:32 +03:00
Ari Koivula b3c15b8f94 Merge branch 'owf' 2014-06-16 16:07:41 +03:00
Laurent Fasnacht 91de92134f Constrain the search not to go under the LCU below if OWF is enabled 2014-06-16 14:27:56 +02:00
Ari Koivula 1c97a10a6d Move intra SAD and SATD functions under strategies. 2014-06-16 12:13:41 +03:00
Laurent Fasnacht f4187dd10c cu_array data structure 2014-06-16 10:54:57 +02:00
Ari Koivula 87ca828a63 Correct intra sad function labels.
- These haven't been 16 bit for a long time.
2014-06-16 10:45:10 +03:00
Laurent Fasnacht 6b408b5904 No-copy works with --no-sao --no-deblock 2014-06-12 11:47:30 +02:00
Laurent Fasnacht 27a49d287d Big refactor to use videoframe, image_list, and image instead of picture* 2014-06-10 09:19:06 +02:00
Laurent Fasnacht 2456c65822 Replace accesses to picture->cu_array with picture_get_cu and picture_get_cu_const 2014-06-05 10:41:58 +02:00
Ari Koivula e585da37e5 Give correct transform depth to RDOQ.
Conflicts:
	src/search.c
2014-05-28 15:47:49 +03:00
Ari Koivula ddc02cc09e Avoid regenerating reference pixels for every rdo mode. 2014-05-22 13:18:28 +03:00
Ari Koivula dbe13d0cba Separate sad intra search from rdo search. 2014-05-22 12:47:45 +03:00
Ari Koivula 19ce21e07c Split final cost to luma and chroma functions. 2014-05-22 09:45:00 +03:00
Ari Koivula a6962e2974 Separate intra transform coding to luma and chroma functions. 2014-05-22 09:40:34 +03:00
Ari Koivula f9a603e4ea Move intra mode search form intra module to search module.
- Make the actual intra prediction function global.

- Move the rdo stuff to rdo module.
2014-05-19 16:12:02 +03:00
Ari Koivula 656b0a321b Add chroma mode to lcu_set_intra_mode.
- This is needed for intra split.
2014-05-19 15:58:54 +03:00
Ari Koivula 921f58b249 Add tr_split to lcu_set_intra_mode. 2014-05-19 15:58:54 +03:00
Ari Koivula 846b608125 Add transform split recursion to intra reconstruction. 2014-05-19 15:58:54 +03:00
Ari Koivula c9a8756fbd Fix NxN scan mode for lcu_get_final_cost.
- Scan mode was always selected according to the first PU mode.
2014-05-15 16:20:35 +03:00
Marko Viitanen b08047cce9 Fixed intra chroma mode selection 2014-05-15 09:50:05 +03:00
Tapio Katajisto 4d879945b2 Fixed cost calculations in fme 2014-05-15 03:42:42 +00:00
Ari Koivula f0e990905e Remove chroma mode "36".
- It's an unnecessary chore to handle this special case everywhere (it means
  chroma_mode == intra_mode). Better just to use the actual mode.
2014-05-14 19:56:35 +03:00
Laurent Fasnacht bb86f24000 Take advantage of the new buffers to remove uneeded item assignment 2014-05-14 13:27:13 +02:00
Laurent Fasnacht 6607c9f563 Use new buffers for search 2014-05-14 13:27:12 +02:00
Laurent Fasnacht c257c4b863 Add const for the buffers 2014-05-14 13:27:12 +02:00
Ari Koivula 9147b7acbf Split residual quantization to separate luma and chroma function. 2014-05-14 11:19:48 +03:00
Tapio Katajisto cc92cfee18 Added few warnings to Makefile
Cleaned fme code a bit
2014-05-14 01:49:34 +00:00
Tapio Katajisto efc43c8b3a Added fractional pixel motion estimation
Added farctional mv support for inter recon

Added 1/8-pel chroma and 1/4-pel luma interpolation
2014-05-14 01:42:02 +00:00
Ari Koivula 1c38209cab Add missing include. 2014-05-13 09:33:05 +03:00
Laurent Fasnacht 0e6f1c99fc Refactor picture to remove hidden dependency between slice and tiles
picture.type -> encoder_state->global->pictype
picture.slicetype -> encoder_state->global->slicetype
picture.slice_sao_luma_flag -> 1 (was constant)
picture.slice_sao_chroma_flag -> 1 (was constant)

This may be changed later. For now it's better to avoid having slice related stuff in picture.
2014-05-07 11:55:48 +02:00
Laurent Fasnacht 6c6adf18c7 Refactor encoder_state 2014-05-07 11:47:31 +02:00
Ari Koivula bdc16d2612 Improve cu_info coded block flag data structure a bit.
- It works just like the old structure except that the flags are checked with
  bitmasks instead of having the flag value be propagated upwards. There isn't
  really any benefit to this because the flags still have to be propagated to
  parent CUs.

- Wrapped them inside a struct to make copying them easier. (Just need to copy
  the struct instead of making individual copies)
2014-05-06 18:28:04 +03:00
Ari Koivula 4490e8afd6 Remove depth dimension from picture->cu_array.
- It isn't used for anything anymore.

- It was used in the past to hold information during search, but now that
  information is held in lcu_t structs.
2014-04-28 10:18:22 +03:00
Laurent Fasnacht 5e7945888a Inter-frame prediction with tiles works.
Many thanks to Jean-Hugues Recolin for the insightful comments about shifts!
2014-04-25 09:28:00 +02:00
Laurent Fasnacht 571a373f69 Use tile offset in search 2014-04-24 08:38:24 +02:00
Laurent Fasnacht 19b1642aa2 Removed all cabac parameters (cabac is part of encoder_state) 2014-04-22 11:46:53 +02:00
Laurent Fasnacht 5fea5875a5 Huge refactoring
Split some parts of encoder_control into encoder_state
(idea: encoder_control is immutable)

Goal is to allow multiple substreams in the future.
2014-04-22 10:39:12 +02:00
Laurent Fasnacht 3396264f3c Moved g_cur_lambda_cost into encoder_control.cur_lambda_cost 2014-04-17 12:00:21 +02:00
Laurent Fasnacht 534013be77 Remove g_lambda_cost 2014-04-17 11:49:27 +02:00
Laurent Fasnacht 4a9c239027 Remove g_bitdepth 2014-04-17 11:13:13 +02:00
Laurent Fasnacht 7a2b883059 Remove encoder_input width, height, height_in_lcu, and width_in_lcu 2014-04-17 11:13:12 +02:00
Laurent Fasnacht 9ac3b7bf2b encoder->in.cur_pic --> cur_pic 2014-04-17 11:13:10 +02:00
Laurent Fasnacht 52ae027b3a Avoid undefined behavior in memcpy calls
"The memcpy() function shall copy n bytes from the object pointed to by s2 into the object pointed to by s1. If copying takes place between objects that overlap, the behavior is undefined."
2014-04-15 06:30:21 +02:00
Laurent Fasnacht 78c579053a encoder_control should be const in nearly all the code 2014-04-14 10:56:06 +02:00
Ari Koivula 5fa5e01e05 Merge branch 'intra-cleanup'
Conflicts:
	src/intra.c
	src/intra.h
	src/search.c
2014-04-10 13:51:14 +03:00