Commit graph

349 commits

Author SHA1 Message Date
sh4dowb 533d0a1fd4
Added "Read More" button for video descriptions 2021-05-10 00:24:11 -07:00
Samantaz Fox cac5f20a28 Move locales definition to i18n.cr 2021-04-21 16:10:25 +02:00
TheFrenchGhosty beea32661a
Merge pull request #1975 from tenpura-shrimp/disablequic
add option to disable quic
2021-04-17 17:59:16 +00:00
TheFrenchGhosty b9e57d0283
Merge pull request #1977 from syeopite/add996
Change empty home page to search engine like layout
2021-04-17 17:02:47 +00:00
syeopite 5c0458a887
Move navbar search bar render check to templated 2021-04-17 05:18:58 -07:00
Andrew Zhao 9a8f1a0b0a add option to disable quic 2021-04-14 12:03:31 -04:00
syeopite 428747ab69
Allow default_home config to be empty value 2021-04-04 15:20:08 -07:00
syeopite 20b961c1c8
Preserve original cookies 2021-04-02 17:08:55 -07:00
TheFrenchGhosty b794c5cfcf
Set the request cookie to "YES+" 2021-04-01 15:59:24 +00:00
syeopite e08bea5f51
Fix lint 2021-03-31 15:34:29 -07:00
syeopite 739f610507
Add new YT consent cookie to every request 2021-03-31 14:57:00 -07:00
syeopite 608313c1d1
Update regex expressions to handle unexpected '};' 2021-03-29 17:37:12 -07:00
Samantaz Fox aaefa38602 Make the linter happy 2021-03-21 16:05:50 +01:00
Samantaz Fox f99d62a2bc Create youtube API wrapper fo /youtubei/v1/browse 2021-03-21 15:44:18 +01:00
saltycrys 378c0d049e WIP channels fix 2021-03-03 22:21:29 +01:00
Perflyst fc8c555519
Merge pull request #1729 from Rjevski/12factor-database-url-restored
Support "Database URL" in addition to existing DB-related parameters
2021-03-02 18:41:10 +01:00
saltycrys 2bf445e165 Fix playlists with Various Artists author 2021-02-10 01:04:27 +01:00
saltycrys 5f03a583d1 Update code for Crystal 0.36.0
Rename `HTTPClient@socket` to `HTTPClient@io`, see
https://github.com/crystal-lang/crystal/pull/9543.

Rename `URI#full_path` to `URI#request_target`, see
https://github.com/crystal-lang/crystal/pull/10099.
2021-02-02 20:01:02 +01:00
saltycrys 909285ea46
Merge pull request #1732 from saltycrys/unrequire-logger
Unrequire `logger`
2021-01-31 03:48:29 +01:00
saltycrys a0bc0641c7 Unrequire logger
Crystal's `Logger` was required but never used in Invidious. Crystal 0.36.0
removed `Logger` in favor of `Log`.
2021-01-31 03:33:38 +01:00
Andre Borie 79e99908de Allow providing 12-Factor-style Database URL in config 2021-01-30 19:33:58 +00:00
saltycrys c9a316ad35
Merge pull request #1702 from saltycrys/config
Config improvements
2021-01-30 14:46:10 +01:00
Andrew Zhao d0dbbd1cb1 remove https from channel thumbnail in search 2021-01-28 00:51:14 -05:00
saltycrys b45f371911 Make config a constant
Instead of passing around `config` there is now the global `CONFIG`.
2021-01-23 19:39:04 +01:00
saltycrys f1a7ee997b Add config environment variables
The config file can now be specified with `INVIDIOUS_CONFIG_FILE`.
A YAML formatted string can still be passed with `INVIDIOUS_CONFIG`, replacing
the config file.

Additionally all options can now be specified as environment variables.
The syntax for variable names is `INVIDIOUS_` followed by the option name in
upper case. The values are parsed as YAML.

These new env vars only update the provided main configuration, but it is
possible to point the config file at the example config and then use env vars
for all config options:
```
INVIDIOUS_CONFIG_FILE=./config/config.example.yml \
INVIDIOUS_CHANNEL_THREADS=10 \
./invidious
```
2021-01-23 18:58:13 +01:00
HackerNCoder 606dd11b4f
Remove admin_email. Use repos url for captcha ID and reddit header. Add note about not updating changelog 2021-01-07 21:09:24 +01:00
HackerNCoder 4d512d908d
Remove some mentions of omarroth 2021-01-07 19:01:13 +01:00
saltycrys df9e7f284c Adjust log verbosity
The default log level has been changed from `debug` to `info`.
The `debug` log level is now more verbose. `debug` now gives a general overview
of what is happening (where implemented) while `trace` gives all available
details.
2021-01-05 20:51:18 +01:00
saltycrys 6365ee7487 Make logger a constant
Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-05 20:43:19 +01:00
saltycrys 7a8620a570 Add CLI arguments to config file
The log level can now be set with `log_level` (accepts ints and strings).
The log file can now be set with `output` (also accepts `STDOUT`).
2021-01-05 20:35:59 +01:00
Perflyst 2c24bf3222
Merge pull request #1389 from vhuynh3000/decrypt_on_demand
add config to decrypt on demand instead of polling
2021-01-05 19:32:22 +01:00
saltycrys 8b56a038b7 Set content type for HTML error helpers
This fixes `Unexpected char '<' at 1:1` errors caused by content type mismatch.
2021-01-04 05:24:08 +01:00
vhuynh3000 3a2bd4e928 add config to decrypt on demand instead of polling 2021-01-03 20:50:52 +01:00
TheFrenchGhosty 82c8f3b556
Merge pull request #1600 from jksladjflkjsadflkjsadf/closeclients
Close http clients after use
2020-12-30 01:14:31 +00:00
Perflyst b19524d56a
Merge pull request #1609 from saltycrys/add-popular-enabled-option
Add `popular-enabled` option
2020-12-28 11:30:55 +01:00
Andrew Zhao e0d25ff887 Close http clients after using
The crystal http client maintains a keepalive connection to the other
server which stays alive for some time. This should be closed if the
client instance is not used again to avoid hogging resources
2020-12-27 19:40:58 -05:00
saltycrys 198dfffaeb Add popular-enabled option
This is similar to the removed `top-enabled` option but for the Popular feed.
The instance needs to be restarted if the feed was enabled.

Editing admin options on the preferences page is also fixed.

The handling of the feed pages now only happens in a single place.

Instead of redirecting:
  - The Top feed now displays a message that it was removed from Invidious.
  - The Popular feed now displays a message that it was disabled if it was.
2020-12-27 06:12:43 +01:00
Perflyst eeeecf9763
Merge pull request #1572 from saltycrys/add-dash-quality-preference
Add DASH quality preference
2020-12-23 20:36:13 +01:00
saltycrys b39f01dcdf Improve logging
Everything that gets logged now has a log level associated with it.

The log level can be set with the new `-l` or `--log-level` arguments.

The defaul log level is `debug` for now. There aren't many things that get
logged but if the logs get spammed in the future it can be set down to `info`.
2020-12-21 16:32:34 +01:00
saltycrys eea7ca9b72 Add DASH quality preference
The options are `auto` (the current and default behavior), `best` and `worst`.

The UI is only updated once playback starts.
2020-12-19 18:37:30 +01:00
saltycrys 2de206cb81 Fix comments
The YouTube headers are now always added for requests to YouTube.
Previously they were only added for requests going through QUIC.

The session token is now JSON decoded to unescape escaped Unicode characters.

The comment continuation protobuf has been updated and the request now goes
through the YouTube `pbj` JSON API.
2020-12-06 04:11:41 +01:00
saltycrys b409cdece5 Improve error message 2
Electric Boogaloo

The long backtrace has been moved into a `<details>` HTML element, as suggested
by @B0pol. To make the error still visible it has been added to the top under
`Title:`. This also encourages informative issue titles.
2020-12-01 13:41:19 +01:00
Théo Gaillard df3f9a2ae8
fix: channel info parsing 2020-11-30 19:35:45 +01:00
TheFrenchGhosty ca781651cb Fix formatting of #1504 2020-11-30 15:56:01 +01:00
saltycrys 92bb477f68 Improve error message
The error message has been reworded and the issue template now includes the
date, route, version and backtrace.
2020-11-30 13:02:35 +01:00
saltycrys 3dac33ffba
Add backtraces to errors (#1498)
Error handling has been reworked to always go through the new `error_template`,
`error_json` and `error_atom` macros.
They all accept a status code followed by a string message or an exception
object. `error_json` accepts a hash with additional fields as third argument.

If the second argument is an exception a backtrace will be printed, if it is a
string only the string is printed. Since up till now only the exception message
was printed a new `InfoException` class was added for situations where no
backtrace is intended but a string cannot be used.

`error_template` with a string message automatically localizes the message.
Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497
`error_json` with a string message does not localize the message. This is the
same as previous behavior. If translations are desired for `error_json` they
can be added easily but those error messages have not been collected yet.

Uncaught exceptions previously only printed a generic message ("Looks like
you've found a bug in Invidious. [...]"). They still print that message
but now also include a backtrace.
2020-11-30 10:59:21 +01:00
Théo Gaillard b41ca72d2b
revert: remove 'JSON.parse("' 2020-11-26 18:22:31 +01:00
Théo Gaillard 1ba17a0e14
feat: centralize ytInitialData parsing 2020-11-26 13:43:53 +01:00
Théo Gaillard 480d31eb5e
fix: ytInitialData parsing with regex 2020-11-26 07:22:47 +01:00
Émilien Devos 3a2f4d4def
allow to set a custom anti-captcha api url (#1473) 2020-11-20 22:21:26 +01:00
saltycrys 9a8b81fcbe Fix storyboards 2020-10-25 09:35:16 +01:00
Ben Heller 22d9d16a7a
Update parsing of ytInitialData (#1427)
Update parsing of ytInitialData
2020-10-22 10:02:15 +00:00
Matthew McGarvey 0520e67303
[refactor] Finish converting jobs to new job setup (#1420)
[refactor] Finish converting jobs to new job setup
2020-10-17 12:25:57 +00:00
Matthew McGarvey 989317e5d3
Extract RefreshChannelsJob (#1403) 2020-10-15 14:22:41 +00:00
Perflyst 812a21bce6
Merge pull request #1372 from tleydxdy/patch-1
Add exponential backoff to refresh_channels
2020-10-10 14:01:47 +02:00
Perflyst f65e57ee82
Fix formatting 2020-10-03 15:19:12 +02:00
Omar Roth 9995a8d53f
Allow cookies from anti-captcha to be optional (#1371) 2020-09-09 23:04:08 +00:00
tleydxdy f02ac01f7e
Add exponential backoff to refresh_channels
In practice with the patch I usually see backoff to 2 hours when blocked, so it should improve recovery time. The lim_thread is to work with multi-threading, not sure if it's the best way to do it.
2020-09-09 11:47:11 -04:00
Ben Heller 4a6e920d0e
Use new youtube API to fetch channel videos (#1355)
* Use new API to fetch videos from channels

This mirrors the process used by subscriptions.gir.st. The old API is
tried first, and if it fails then the new one is used.

* Use the new API whenever getting videos from a channel

I created the get_channel_videos_response function because now instead
of just getting a single url, there are extra steps involved in getting
the API response for channel videos, and these steps don't need to be
repeated throughout the code.

The only remaining exception is the bypass_captcha function, which still
only makes a request to the old API. I don't know whether this code
needs to be updated to use the new API for captcha bypassing to work
correctly.

* Correctly determine video length with new api

* Remove unnecessary line
2020-09-02 20:28:57 +00:00
Omar Roth db83ede73c
Add missing default for ConfigPreferences 2020-07-26 11:09:45 -04:00
Omar Roth 452d1e8307
Fix warnings with latest version of Crystal 2020-07-26 10:59:31 -04:00
Omar Roth b5f79444f9
Fix optional value in data extractor 2020-07-21 20:11:13 -04:00
Omar Roth 8c65b8c16f
Update playlists to support polymer redesign 2020-06-24 22:19:36 -04:00
Omar Roth 83b1d80a5b
Fix signature URL regex 2020-06-16 19:21:29 -05:00
Omar Roth 056e7432bd
Update channel playlists to use polymer 2020-06-16 17:51:35 -05:00
Omar Roth 1eca969cf6
Add support for polymer redesign 2020-06-15 18:18:04 -05:00
Omar Roth c1cbdae5ee
Make HOST_URL constant 2020-06-15 18:13:14 -05:00
Omar Roth 4d4b6a2fa0
Remove top page 2020-06-15 18:11:36 -05:00
Omar Roth d30a972a90
Support Crystal 0.35.0 2020-06-15 18:11:06 -05:00
Omar Roth 6435c7b921
Fix reCaptcha 2020-05-28 12:21:47 -05:00
Omar Roth ceb252986e
Update captcha job 2020-05-25 12:52:15 -05:00
Omar Roth 750ef296c6
Update captcha handler 2020-05-13 16:09:39 -05:00
Omar Roth ca1185d0be
Fix warnings in latest version of Crystal 2020-04-09 12:18:09 -05:00
Omar Roth 02d4186b11
Fix player matching 2020-04-09 10:55:50 -05:00
Omar Roth 3f97bebd69
Support adding video to playlist from watch page 2020-04-09 10:55:32 -05:00
Omar Roth 59a15ceef6
Remove VarInt class 2020-03-15 17:47:16 -04:00
Omar Roth 4011a113cc
Strip invalid characters from referer URLs 2020-03-15 17:47:16 -04:00
Omar Roth a117d87f33
Skip validation checks for videoplayback, ggpht 2020-03-04 13:06:17 -05:00
leonklingele 0d536d11e3
Verify token signature in constant time, Run cheap checks first in token validation process (#1032)
* Verify token signature in constant time

To prevent timing side channel attacks

* Run cheap checks first in token validation process

Expensive checks such as the nonce lookup on the database or the
signature check can be run after cheap/fast checks.
2020-03-02 10:04:36 -06:00
Omar Roth 43da06a354
Remove temp fix for crystal/crystal-lang#7383 2020-02-20 18:30:46 -05:00
Leon Klingele bc9dc3bf1e
Update code formatting for Crystal 0.33.0
Crystal 0.33.0 introduced some changes to to the code formatter.
Run "crystal tool format" so CI doesn't fail anymore.
2020-02-15 19:52:28 +01:00
Omar Roth a2c5211b20
Check /browse_ajax for channel blocks 2020-02-01 11:23:12 -05:00
Omar Roth 648cc0f006
Refactor signature extraction 2020-01-24 17:02:28 -05:00
Omar Roth 823603650f
Add support for /sorry/index CAPTCHA 2019-12-03 19:14:11 -05:00
Omar Roth a017574f74 Add support for force_resolve to QUIC client 2019-11-28 08:19:28 -06:00
Omar Roth 0e3a48ff76
Update QUICPool 2019-11-24 13:41:47 -05:00
Omar Roth d46b26e3bc
Use QUIC for connections to YouTube 2019-11-18 17:28:32 -05:00
Omar Roth 71bc9eea28
Add support for Anti-Captcha 2019-11-09 14:22:39 -05:00
Omar Roth 142d974641
Use force_resolve for search suggestions 2019-11-07 12:25:34 -05:00
Omar Roth bcdb8cd770
Fix default fo dark_mode 2019-11-04 17:08:13 -05:00
Omar Roth 833a60f29c
Update pubsub to use client pool 2019-11-01 07:34:36 -04:00
Omar Roth 13e7cca1a4
Bump read timeout 2019-10-28 12:34:50 -04:00
Omar Roth 03e24cccd0
Add support for configurable administrator email 2019-10-27 14:18:07 -04:00
Omar Roth 0cf187dee7
Add support for image captcha in Google login 2019-10-27 00:19:05 -04:00
Omar Roth bdeb325bad
Fix monkeypatch for HTTP::Client 2019-10-26 11:51:23 -04:00
Omar Roth a1225b6d0d
Sanitize input to decode_length_seconds 2019-10-26 10:17:25 -04:00
Omar Roth 202de1436d
Fix broken connections in pool 2019-10-25 23:06:08 -04:00
Omar Roth 7f8746fcd4
Remove invalid connections from pool 2019-10-25 22:40:53 -04:00
Omar Roth e05a25d701
Vary user-agent 2019-10-25 18:02:33 -04:00
Omar Roth 6930570fa2
Add HTTPClient pool 2019-10-25 12:58:16 -04:00