Commit Graph

661 Commits

Author SHA1 Message Date
litetex
f775155d25 Merge pull request #846 from litetex/remove-unused-methods
Remove unused methods
2022-06-19 15:12:15 +02:00
Gábor Lipták
28c9340d69 Correct unit tests 2022-06-18 11:18:38 -04:00
TiA4f8R
287d1dfd63 [SoundCloud] Use the HLS delivery method for all streams and extract only a single stream URL from HLS manifest for MP3 streams
SoundCloud broke the workaround used to get a single file from HLS manifests for Opus manifests, but it still works for MP3 ones.

The code has been adapted to prevent an unneeded request (the one to the Opus HLS manifest) and the HLS delivery method is now used for SoundCloud MP3 and Opus streams, plus the progressive one (for tracks which have a progressive stream (MP3) and for the ones which doesn't have one, it is still used by trying to get a progressive stream, using the workaround).

Streams extraction has been also moved to Java 8 Stream's API and the relevant test has been also updated.
2022-05-29 19:08:18 +02:00
Stypox
b3c620f0d8 Apply code review and Streams rework 2022-05-28 12:00:58 +02:00
TiA4f8R
f7b1515290 [YouTube] Refactor DASH manifests creation
Move DASH manifests creation into a new subpackage of the YouTube package, dashmanifestcreators.
This subpackage contains:

- CreationException, exception extending Java's RuntimeException, thrown by manifest creators when something goes wrong;
- YoutubeDashManifestCreatorsUtils, class which contains all common methods and constants of all or a part of the manifest creators;
- a manifest creator has been added per delivery type of YouTube streams:
  - YoutubeProgressiveDashManifestCreator, for progressive streams;
  - YoutubeOtfDashManifestCreator, for OTF streams;
  - YoutubePostLiveStreamDvrDashManifestCreator, for post-live DVR streams (which use the live delivery method).

Every DASH manifest creator has a getCache() static method, which returns the ManifestCreatorCache instance used to cache results.

DeliveryType has been also extracted from the YouTube DASH manifest creators part of the extractor and moved to the YouTube package.

YoutubeDashManifestCreatorTest has been updated and renamed to YoutubeDashManifestCreatorsTest, and YoutubeDashManifestCreator has been removed.

Finally, several documentation and exception messages fixes and improvements have been made.
2022-05-28 12:00:56 +02:00
Stypox
cfc13f4a6f [YouTube] Reduce exception generation code and move several attributes of MPD documents into constants
Rename YoutubeDashManifestCreationException to CreationException.
Also use these constants in YoutubeDashManifestCreatorTest.
2022-05-28 12:00:53 +02:00
Stypox
5c83409039 [YouTube] Rewrite manifest test and rename long methods 2022-05-28 12:00:51 +02:00
TiA4f8R
436ddde29f Use assertThrows in YoutubeDashManifestCreatorTest 2022-05-28 12:00:48 +02:00
TiA4f8R
d64d7bbd01 Move ManifestCreatorCache tests to a separate class and remove override of equals and hashCode methods in ManifestCreatorCache
These methods don't need to be overriden, as they are not excepted to be used in collections.
Also improve the toString method of this class, which contains also now clearFactor and maximumSize attributes and for each operations.
2022-05-28 12:00:48 +02:00
TiA4f8R
2fb1a412a6 Fix Checkstyle issues, revert resolution string changes for YouTube video streams and don't return the rn parameter in DASH manifests
This parameter is still used to get the initialization sequence of OTF and POST-live streams, but is not returned anymore in the manifests.
It has been removed in order to avoid fingerprinting based on the number sent (e.g. when starting to play a stream close to the end and using 123 as the request number where it should be 1) and should be added dynamically by clients in their requests.
The relevant test has been also updated.

Checkstyle issues in YoutubeDashManifestCreator have been fixed, and the changes in the resolution string returned for video streams in YoutubeStreamExtractor have been reverted, as they create issues on NewPipe right now.
2022-05-28 12:00:47 +02:00
TiA4f8R
aa4c10e751 Improve documentation and adress most of the requested changes
Also fix some issues in several places, in the code and the documentation.
2022-05-28 12:00:46 +02:00
TiA4f8R
6985167e63 Add tests for YoutubeDashManifestCreator and ManifestCreatorCache
The test added in YoutubeDashManifestCreator uses a video of the Creative Commons channel, licenced under the Creative Commons Attribution licence (reuse allowed).
Also remove public keywords of tests in UtilsTest, as suggested by SonarLint, because they are not needed with Junit 5.
2022-05-28 12:00:46 +02:00
TiA4f8R
f6ec7f9a61 Update DefaultStreamExtractorTest and SoundcloudStreamExtractorTest to support changes made in Stream classes 2022-05-28 12:00:45 +02:00
litetex
2015eb374a Removed more unused methods 2022-05-09 21:05:03 +02:00
litetex
f69b0ff77b Remove unused methods 2022-05-09 20:59:25 +02:00
Fynn Godau
c38c016de5 Bandcamp: extract stream length 2022-04-24 21:24:19 +02:00
Nickoriginal
b5f3f9eb90 Update USER_AGENT to match the latest Firefox 2022-04-24 00:27:23 +03:00
TiA4f8R
b30e341559 Add link to learn more about the issue which makes YoutubeStreamExtractorRelatedMixTest.testRelatedItems test disabled 2022-04-04 20:36:59 +02:00
TiA4f8R
70812fa611 Update YouTube stream mocks and disable YoutubeStreamExtractorRelatedMixTest
Mixes seems to be not given by YouTube anymore if you use a PENDING consent cookie value.
As mocks needs to updated, the test is always failing because of this change.
2022-04-02 19:28:49 +02:00
TiA4f8R
9ca647a750 Update mocks 2022-03-27 22:10:59 +02:00
TiA4f8R
3d38459cf3 [YouTube] Reduce InnerTube response sizes by adding the prettyPrint parameter with the false value
InnerTube responses return pretty printed responses, which increase responses' size for nothing.

By using the prettyPrint parameter on requests and setting its value to false, responses are not pretty printed anymore, which reduces responses size, and so data transfer and processing times.
This usage has been recently deployed by YouTube on their websites.
2022-03-27 20:52:40 +02:00
litetex
349ba8db7f Improve tests and randomness
- Use the existing RNG inside YoutubeParsingHelper
- Deduplicated test-setup for YouTube tests
- Minor improvements
2022-03-27 20:52:38 +02:00
TiA4f8R
52376949e5 Add setSeedForVideoTests method in YoutubeStreamExtractor tests
In order to use still use mocks with the generation of random strings in player requests, we need to use YoutubeParsingHelper.setSeedForVideoTests() method in every stream test.
2022-03-27 20:51:39 +02:00
litetex
29408f9356 `YoutubeMusicSearchExtractorTest$Suggestion` has no mock data 2022-03-26 20:52:28 +01:00
litetex
fcee247f9f Switched to mockonly tests
* Made the reason why the tests are mockonly mandatory
2022-03-26 20:52:28 +01:00
litetex
0fceb4686b Added missing mock data 2022-03-26 20:52:27 +01:00
litetex
70b20f0d6b Updated mock data after conflicts 2022-03-26 20:52:27 +01:00
litetex
6a6c9359af Fixed kurzgesagt test - They changed their description... 2022-03-26 20:52:27 +01:00
litetex
26596215fa Refactored `MediaCCCRecentListExtractorTest` 2022-03-26 20:52:26 +01:00
litetex
53962bfd7d Disabled suggestion test for now
Has to be done once YT's backed works again...
2022-03-26 20:52:26 +01:00
litetex
5d58156cde Update mock data 2022-03-26 20:52:25 +01:00
litetex
164e21b5af Fixed `MediaCCCRecentKiosk`
Ignore faulty data/items (with duration <= 0)
2022-03-26 20:52:23 +01:00
litetex
49dfdae5d2 Removed useless throw declaration 2022-03-26 20:51:38 +01:00
litetex
69a58fd3d1 Don't use sysout 2022-03-26 20:51:37 +01:00
litetex
2bd4299563 Fixed test: Peertube - Use new comments and videos
Old comments/videos got deleted
2022-03-26 20:49:39 +01:00
litetex
606a386a98 Fix tests: What's peertube has less comments now 2022-03-26 20:49:39 +01:00
litetex
4d1a1c8fb8 Better test for `MediaCCCRecentListExtractorTest`
* Use assertAll
* Show which item is affected
2022-03-26 20:49:38 +01:00
litetex
ba43dbaa28 Fix Tests: The channel lost abos 2022-03-26 20:49:38 +01:00
litetex
ff436e5740 Fixed test: YT mix playlist 2022-03-26 20:49:38 +01:00
Stypox
09ddb6adbb [YouTube] Add MockOnly to method testing mixes in related items 2022-03-19 10:54:38 +01:00
Stypox
8201b3b90e [YouTube] Parse any playlist (including music mixes) in related items 2022-03-19 10:48:13 +01:00
Stypox
13f7900816 [YouTube] Add test for genre mix 2022-03-19 10:48:13 +01:00
Stypox
279f3a20fe [YouTube] Fix mix tests with invalid video ids
Replaces mix tests based on a strange mix type RDQM{videoId} (only reference I could find is https://github.com/ytdl-org/youtube-dl/issues/26228) and with an invalid video id of 13 characters (the first two characters were QM, but even after removing QM there still wasn't a video available at that id).
Also updates mocks.
2022-03-19 10:48:13 +01:00
Stypox
d660c04838 [YouTube] Also test playlist type in playlist tests 2022-03-19 10:48:13 +01:00
Stypox
34a4484c72 [YouTube] Add test for a video with a mix in related items 2022-03-19 10:44:06 +01:00
Stypox
d8f2031619 Merge pull request #816 from Stypox/mock-only-extension
Add `@MockOnly` Junit 5 extension
2022-03-19 10:40:38 +01:00
litetex
cc2e4d7104 Merge pull request #815 from litetex/fix-soundcloud-id-once-and-for-all
Removed hardcoded soundcloud HARDCODED_CLIENT_ID
2022-03-17 13:54:08 +01:00
TiA4f8R
35e082248e Fix YouTube and SoundCloud playlists tests 2022-03-16 19:40:30 +01:00
Stypox
73d1fd472f Add MockOnly junit 5 test extension 2022-03-16 19:03:08 +01:00
Stypox
ef71a5fa0f static final instead of final static 2022-03-16 17:24:33 +01:00