Commit graph

1003 commits

Author SHA1 Message Date
Émilien Devos 12b46bbd41
switch to innertube API for about channels (#2255) 2021-08-12 16:00:26 -07:00
Émilien Devos 88c5e3b6fa
Use the new youtube api for comments (#2217)
* use the new youtube api for comments
* remove PG_DB & action parameter + allow force region
* support new comments data with onResponseReceivedEndpoints
2021-08-12 19:14:30 +02:00
syeopite 1321c90920
Extract channel routes (#2227)
* Extract primary channel routes from invidious.cr

Also removes timedtext_video stub since all it does is redirect to the
homepage. However, Invidious's 404 handler already does this.

--
As the template for the channel about page doesn't exist yet, the
behavior for the /channel/:ucid/about endpoint has been changed to be
the same as what's currently present on Invidious

(cherry picked from commit 8fad19d8057d7d22e3de27ebbc88a9978c1df27b)

* Manually extract brand_redirect from 1b569bbc99207cae7c20aa285f42477ae361dd30

This commit manually extracts the brand_redirect function from the
commit mentioned.

However, the redirect to the  `.../about` endpoint is removed due to the
fact that it doesn't exist yet.

This commit is also mainly just a bridge for the next few cherry picks from
\#2215

* Update brand_redirect to use youtubei resolve_url

(cherry picked from commit 53335fe7cfdfac392365b7cac447bc7cc6478134)

* Add additional channel endpoints to brand_redirect

(cherry picked from commit 8fc6f3add637dabb09b2034f4d82fc3d039ba15c)

* Add separate handler for /profile endpoint

* Add /channel/:ucid/home route

* Document all channel brand_urls
2021-08-03 23:46:15 +02:00
syeopite 3e5c353298
Merge pull request #2205 from syeopite/fix-age-restricted-videos
Partial (and temporary) fix for age restricted videos
2021-07-14 10:11:03 -07:00
syeopite ae61662f61
Debloat channels.cr into multiple files (#2225)
Cherry picked from ui overhaul branch with a few modifications:
  - channel folder is renamed to channels
  - parsing for channel home and featured channels are removed due to
    lack of infrastructure from other commits

(cherry picked from commit 44d18b8e147b47ad06a54cc6fd08423d9f39074d)
2021-07-14 17:46:12 +02:00
syeopite 54b19a04bb
Fix caption parsing on age restricted videos 2021-06-27 08:35:28 -07:00
Samantaz Fox 5a8825d016
Fix quoting of 'none' in CSP header
The keyword 'none' must be surrounded by single quotes.
Regression introduced by #2168.
2021-06-20 18:43:00 +02:00
syeopite 31fa3dfd59
Add route to redirect to another instance 2021-06-19 04:16:17 -07:00
TheFrenchGhosty b8f0b4b583
Typo 2021-06-19 09:40:33 +02:00
Samantaz Fox 42d9fd9c88 Rewrite response headers
Fixes #2018 and #2153
2021-06-17 19:53:39 +02:00
syeopite 0a87ba6930
Decode URL encoded preference cookie when parsing 2021-06-08 13:11:04 -07:00
syeopite 065c104f27
Upgrade to crystal 1.0.0 2021-06-08 13:10:58 -07:00
Samantaz Fox cac5f20a28 Move locales definition to i18n.cr 2021-04-21 16:10:25 +02:00
Andrew Zhao 9a8f1a0b0a add option to disable quic 2021-04-14 12:03:31 -04:00
syeopite 877bb5c821
Add aliases for watch route 2021-04-06 20:36:23 -07:00
Svallinn cbdba66ef3
Use the youtubei API over the legacy one 2021-03-24 05:33:45 +00:00
Andrew Zhao 89fd35e02d fix comment replies 2021-03-20 00:43:12 -04:00
Svallinn fa050fb8a8
Fix: functional hl / dark_mode / thin_mode params 2021-03-17 19:09:37 +00:00
TheFrenchGhosty 3cc9114f81
Merge pull request #1829 from SimonPhoenix96/xml-json-yt-sub-import
integrate legacy yt xml subscription import with new json sub import
2021-03-05 01:01:32 +01:00
Perflyst fc8c555519
Merge pull request #1729 from Rjevski/12factor-database-url-restored
Support "Database URL" in addition to existing DB-related parameters
2021-03-02 18:41:10 +01:00
simonphoenix96 84487b2e52 format invidious.cr using crystal cli 2021-02-27 21:59:09 +01:00
Simon Phoenix 9689376de6
fixed line indent typo 2021-02-27 19:12:01 +01:00
simonphoenix96 a2f79a163f integrate legacy yt xml subscription import with new json sub import 2021-02-27 18:58:55 +01:00
Andrew Zhao 2600695927 fix search
Put search page in the super secret proto field
2021-02-25 22:19:22 -05:00
TheFrenchGhosty c451aab150
Enable the Hebrew translation since it's now translated at more than 80% 2021-02-14 19:37:46 +00:00
TheFrenchGhosty 48bff9a5d2
Enable the Indonesian translation since it's now translated at more than 80% 2021-02-14 19:35:05 +00:00
saltycrys 5f03a583d1 Update code for Crystal 0.36.0
Rename `HTTPClient@socket` to `HTTPClient@io`, see
https://github.com/crystal-lang/crystal/pull/9543.

Rename `URI#full_path` to `URI#request_target`, see
https://github.com/crystal-lang/crystal/pull/10099.
2021-02-02 20:01:02 +01:00
saltycrys 991a04dc2a Adjust routes
Simple routes have been moved into a single `Misc` file.

Embed routes have been moved into a single `Embed` file.

The preferences route has been renamed to be more consistent with other parts
of the codebase.
2021-02-02 06:18:16 +01:00
Perflyst 82da5cfd01
Merge pull request #1712 from tenpura-shrimp/bumpvideojs
Bump videojs and fix webworker
2021-01-31 12:18:40 +01:00
Andre Borie 79e99908de Allow providing 12-Factor-style Database URL in config 2021-01-30 19:33:58 +00:00
TheFrenchGhosty bd893cb24c Enable the Finish translation since it's now translated at more than 80% 2021-01-30 16:42:30 +01:00
saltycrys c9a316ad35
Merge pull request #1702 from saltycrys/config
Config improvements
2021-01-30 14:46:10 +01:00
Andrew Zhao 4a0b10984a Bump videojs and fix webworker 2021-01-29 19:08:28 -05:00
saltycrys 70e14f92a4 Only start refresh jobs when necessary
If `channel_threads` or `feed_threads` is set to zero the corresponding job is
now not started.
2021-01-23 19:45:42 +01:00
saltycrys b45f371911 Make config a constant
Instead of passing around `config` there is now the global `CONFIG`.
2021-01-23 19:39:04 +01:00
saltycrys f1a7ee997b Add config environment variables
The config file can now be specified with `INVIDIOUS_CONFIG_FILE`.
A YAML formatted string can still be passed with `INVIDIOUS_CONFIG`, replacing
the config file.

Additionally all options can now be specified as environment variables.
The syntax for variable names is `INVIDIOUS_` followed by the option name in
upper case. The values are parsed as YAML.

These new env vars only update the provided main configuration, but it is
possible to point the config file at the example config and then use env vars
for all config options:
```
INVIDIOUS_CONFIG_FILE=./config/config.example.yml \
INVIDIOUS_CHANNEL_THREADS=10 \
./invidious
```
2021-01-23 18:58:13 +01:00
TheFrenchGhosty 799f97e847 Make invidious use all the translation files 2021-01-23 18:07:55 +01:00
Andrew Zhao b7fe1db89a save host when using dash manifest 2021-01-10 18:00:45 -05:00
saltycrys c81ca187f8 Fix downloads
The `itag` is now converted to a number, matching the `itag` of
`Video.adaptive_fmts` and `Video.fmt_stream`.
2021-01-09 20:40:01 +01:00
saltycrys b0b8ba7000 Respect use_pubsub_feeds config
Setting `use_pubsub_feeds: false` now properly disables it.
2021-01-07 20:52:32 +01:00
saltycrys 6365ee7487 Make logger a constant
Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-05 20:43:19 +01:00
saltycrys 7a8620a570 Add CLI arguments to config file
The log level can now be set with `log_level` (accepts ints and strings).
The log file can now be set with `output` (also accepts `STDOUT`).
2021-01-05 20:35:59 +01:00
Perflyst 2c24bf3222
Merge pull request #1389 from vhuynh3000/decrypt_on_demand
add config to decrypt on demand instead of polling
2021-01-05 19:32:22 +01:00
saltycrys 36e9fb9d68 Fix watch_videos endpoint
Playlists created by `watch_videos` do not have an author which caused a crash
previously.
2021-01-04 05:35:59 +01:00
vhuynh3000 3a2bd4e928 add config to decrypt on demand instead of polling 2021-01-03 20:50:52 +01:00
Perflyst c89632d2a8
Merge pull request #1608 from saltycrys/add-subscription-traces
Add Subscription Traces
2020-12-31 11:30:04 +01:00
TheFrenchGhosty 82c8f3b556
Merge pull request #1600 from jksladjflkjsadflkjsadf/closeclients
Close http clients after use
2020-12-30 01:14:31 +00:00
Perflyst b19524d56a
Merge pull request #1609 from saltycrys/add-popular-enabled-option
Add `popular-enabled` option
2020-12-28 11:30:55 +01:00
Andrew Zhao e0d25ff887 Close http clients after using
The crystal http client maintains a keepalive connection to the other
server which stays alive for some time. This should be closed if the
client instance is not used again to avoid hogging resources
2020-12-27 19:40:58 -05:00
saltycrys 198dfffaeb Add popular-enabled option
This is similar to the removed `top-enabled` option but for the Popular feed.
The instance needs to be restarted if the feed was enabled.

Editing admin options on the preferences page is also fixed.

The handling of the feed pages now only happens in a single place.

Instead of redirecting:
  - The Top feed now displays a message that it was removed from Invidious.
  - The Popular feed now displays a message that it was disabled if it was.
2020-12-27 06:12:43 +01:00
saltycrys c4ef055248 Add RefreshChannelsJob traces
Traces can be enabled with `-l trace`.

The problem with subscriptions is that sometimes requests to YouTube never
finish. As soon as that happens `channel-threads` times subscriptions stop
being refreshed. This is most likely a problem with the lsquick bindings.
2020-12-27 05:20:33 +01:00
Perflyst 8df591e8d4
Merge pull request #1595 from saltycrys/improve-storyboards
Improve storyboards
2020-12-23 14:30:10 +01:00
saltycrys 75c6844b66 Improve storyboards
Instead of limiting the width of storyboards through CSS it is now done in the
VTT directly.
2020-12-21 17:02:36 +01:00
saltycrys b39f01dcdf Improve logging
Everything that gets logged now has a log level associated with it.

The log level can be set with the new `-l` or `--log-level` arguments.

The defaul log level is `debug` for now. There aren't many things that get
logged but if the logs get spammed in the future it can be set down to `info`.
2020-12-21 16:32:34 +01:00
Matthew McGarvey fb06482916 Extract out User Preferences endpoints 2020-12-15 18:54:30 -06:00
Amanda Graven 848add1b1b
Set channel thumbnail as icon for feeds (#1106) 2020-12-14 19:25:39 +01:00
Matthew McGarvey 2dacdf0210 Extract login/signout routes from global file 2020-12-08 20:50:39 -06:00
Andrew Zhao 0cb442d40e do not proxy hls on supported browser 2020-12-08 00:53:48 -05:00
Lukas 62e8c09183 Fix YouTube subscription import parser 2020-12-06 21:47:50 +01:00
saltycrys 4184fb3ae7
Increase YouTube request timeout (#1540)
Increase YouTube request timeout
2020-12-06 12:12:02 +00:00
Sonic-Y3k 03d0b296e1
Fix invidious version for docker image
As Alpine-Linux per default uses the busybox date command, the -f option is not available and the build will fail.
2020-12-05 20:06:24 +01:00
saltycrys adccca366e Fix invidious version for old git versions
The `%cs` format was only added to git in version 2.25 while `%ci` has been
around forever.
2020-12-04 01:58:27 +01:00
Matthew McGarvey d7377015a2 Extracting search endpoints 2020-12-01 18:02:39 -06:00
saltycrys 2478e67d16 Switch to date based versioning scheme
Since no new tags are created for releases the version has been frozen for some
time, with only the commit hash changing.

Versions based on the latest commit date make it much easier to identify them.
2020-12-01 13:29:30 +01:00
saltycrys 3dac33ffba
Add backtraces to errors (#1498)
Error handling has been reworked to always go through the new `error_template`,
`error_json` and `error_atom` macros.
They all accept a status code followed by a string message or an exception
object. `error_json` accepts a hash with additional fields as third argument.

If the second argument is an exception a backtrace will be printed, if it is a
string only the string is printed. Since up till now only the exception message
was printed a new `InfoException` class was added for situations where no
backtrace is intended but a string cannot be used.

`error_template` with a string message automatically localizes the message.
Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497
`error_json` with a string message does not localize the message. This is the
same as previous behavior. If translations are desired for `error_json` they
can be added easily but those error messages have not been collected yet.

Uncaught exceptions previously only printed a generic message ("Looks like
you've found a bug in Invidious. [...]"). They still print that message
but now also include a backtrace.
2020-11-30 10:59:21 +01:00
Matthew McGarvey 192d2b86b6 Extract playlist routes from global file 2020-11-19 19:21:48 -06:00
Matthew McGarvey dda02f783f Extract embed routes into separate classes 2020-11-06 16:44:36 -06:00
Perflyst cda322e311
Merge pull request #1449 from raycheung/master
Fix stale playlist video counting on add/drop (#1448)
2020-11-03 17:39:34 +01:00
Matthew McGarvey 2cded8dcac Extract /watch route from main file 2020-11-02 18:06:06 -06:00
Ray Cheung babe0c31ad Fix stale playlist video counting on add/drop (https://github.com/iv-org/invidious/issues/1448)
The return of `cardinality()` is not affected by the `array_append()`
and `array_remove()` of the same statement.
2020-11-01 12:20:44 +08:00
saltycrys 9a8b81fcbe Fix storyboards 2020-10-25 09:35:16 +01:00
Matthew McGarvey 0520e67303
[refactor] Finish converting jobs to new job setup (#1420)
[refactor] Finish converting jobs to new job setup
2020-10-17 12:25:57 +00:00
Matthew McGarvey d571573e52
Set preferences before conditional breaks in before_all (#1412) 2020-10-16 10:23:18 +00:00
Matthew McGarvey 989317e5d3
Extract RefreshChannelsJob (#1403) 2020-10-15 14:22:41 +00:00
Matthew McGarvey 85c212aee3 Provide rough draft of better project organization 2020-10-05 23:57:19 -05:00
Omar Roth b88872d54b
Fix sub refresh (#1370) 2020-09-09 23:03:27 +00:00
stranger195 13f58d602f
Replace omarroth with iv-org where applicable (#1353)
Fixes #1353
2020-08-25 16:07:16 +00:00
Omar Roth 62f015fc34
Fix playlist export for playlists with more than 100 videos 2020-07-28 17:21:39 -04:00
Omar Roth 452d1e8307
Fix warnings with latest version of Crystal 2020-07-26 10:59:31 -04:00
Omar Roth 92f337c67e
Add playlist import/export 2020-07-26 10:59:31 -04:00
Omar Roth 1eca969cf6
Add support for polymer redesign 2020-06-15 18:18:04 -05:00
Omar Roth c1cbdae5ee
Make HOST_URL constant 2020-06-15 18:13:14 -05:00
Omar Roth 4d4b6a2fa0
Remove top page 2020-06-15 18:11:36 -05:00
Omar Roth d30a972a90
Support Crystal 0.35.0 2020-06-15 18:11:06 -05:00
Dmitry Sandalov 5d8de5fde2
Allow user to subscribe to playlist (#17) 2020-05-17 14:28:00 +03:00
Omar Roth 454ae8656a
Cleanup request headers 2020-05-08 09:00:53 -05:00
Omar Roth 97eb01a28d
Merge weblate 2020-04-20 16:46:00 -05:00
Omar Roth 7bb7003c9d
Fix authorThumbnails in /api/v1/channels 2020-04-10 11:49:51 -05:00
Omar Roth 920463f2ff
Fix playlist_ajax 2020-04-10 11:49:18 -05:00
Omar Roth ca1185d0be
Fix warnings in latest version of Crystal 2020-04-09 12:18:09 -05:00
Omar Roth 3f97bebd69
Support adding video to playlist from watch page 2020-04-09 10:55:32 -05:00
Omar Roth 2e378da922
Add support for Swedish locale 2020-04-04 15:57:29 -05:00
Omar Roth b37f51bd7f
Fix /c/ redirect 2020-04-04 15:31:24 -05:00
Omar Roth 92798abb5d Add manifest-src to CSP 2020-03-19 13:41:08 -05:00
leonklingele 70cbe91776
Migrate to a good Content Security Policy (#1023)
So attacks such as XSS (see [0]) will no longer be of an issue.

[0]: https://github.com/omarroth/invidious/issues/1022
2020-03-15 16:46:08 -05:00
Omar Roth 1443335315
Switch textcaptcha to HTTPS 2020-03-10 11:12:11 -04:00
Omar Roth bb72672dd9
Replace static asset requests with QUIC 2020-03-06 13:53:35 -05:00
Omar Roth d96dee3aa6
Add debug info to videoplayback 2020-03-06 13:50:00 -05:00
Omar Roth bd0aaa343b
Prevent storyboards from hanging 2020-03-05 13:49:06 -05:00
Omar Roth 697c00dccf
Sanitize PLID 2020-02-28 14:10:01 -05:00