a0880c79bb #11: Update channel CHANNELS tab treatment following YouTube-operational-API/issues/121 closure
40e0b35940 Update channel CHANNELS tab treatment following YouTube-operational-API/issues/121 closure
Benjamin_Loison commented on issue Benjamin_Loison/YouTube_captions_search_engine#7 2023-01-16 15:46:04 +01:00
Add multi-threading

Previous commit removes the bug of having following output when doing:

unzip -l UCWeg2Pkate69NFdBeuRFTAw.zip
   147928  2023-01-14 10:15   99997.json
   ...
---------     …
Benjamin_Loison commented on issue Benjamin_Loison/YouTube_captions_search_engine#11 2023-01-15 15:27:17 +01:00
Current algorithm doesn't check all channel tabs

Note that we could also give a try to YouTube Data API v3 endpoints Search: list with relatedToVideoId filter and Videos: list with part=suggestions using OAuth (note that it is maybe to some…

Benjamin_Loison opened issue Benjamin_Loison/YouTube_captions_search_engine#28 2023-01-15 15:01:23 +01:00
Make this algorithm able to completely treat the YouTube graph
10c5c1d605 #11: Add the treatment of channels' tab, but only postpone unlisted videos treatment
Benjamin_Loison commented on issue Benjamin_Loison/YouTube_captions_search_engine#11 2023-01-15 14:37:09 +01:00
Current algorithm doesn't check all channel tabs

Note that we could find even more channels/videos by analyzing user messages etc however as we don't have any guarantee that a message contains a channel or a video, we don't treat them. So we…

51a70f6e54 #7: Make commentsCount and requestsPerChannel compatible with multithreading
2ff8234009 #7: Make commentsCount and requestsPerChannel compatible with multithreading
aa97c94bf8 #11: Add a first iteration for the CHANNELS retrieval
Benjamin_Loison commented on issue Benjamin_Loison/YouTube_captions_search_engine#2 2023-01-15 01:42:32 +01:00
Add data logging

As a practical experience, with 4 threads, we treat the 50 most subscribers French channels in 24 hours.

d1b84335d1 #11: Add --youtube-operational-api-instance-url parameter and use exit(EXIT_{SUCCESS, FAILURE}) instead of exit({0, 1})
2a3b902b95 #1&: Add --youtube-operational-api-instance-url parameter and use exit(EXIT_{SUCCESS, FAILURE}) instead of exit({0, 1})
Benjamin_Loison commented on issue Benjamin_Loison/YouTube_captions_search_engine#2 2023-01-14 15:48:10 +01:00
Add data logging

If was having a lot of storage compared to the typical execution of the algorithm, using multiple computers to run this YouTube graph discovery would make sense. Should then also pay attention to…

Benjamin_Loison commented on issue Benjamin_Loison/YouTube_captions_search_engine#11 2023-01-14 15:36:15 +01:00
Current algorithm doesn't check all channel tabs

Should make another boolean USE_YT_LEMNOSLIFE_COM_YOUTUBE_OPERATIONAL_API_INSTANCE like [USE_YT_LEMNOSLIFE_COM_NO_KEY_SERVICE](https://gitea.lemnoslife.com/Benjamin_Loison/YouTube_captions_sear…

Benjamin_Loison commented on issue Benjamin_Loison/YouTube_captions_search_engine#12 2023-01-14 15:26:31 +01:00
Concerning the videos, we could quickly get rid of channels not having videos by using videos?part=statistics

As mentioned in this comment of #11, we want to do perfect retrieving, so using `videos?par…

Benjamin_Loison closed issue Benjamin_Loison/YouTube_captions_search_engine#12 2023-01-14 15:26:31 +01:00
Concerning the videos, we could quickly get rid of channels not having videos by using videos?part=statistics
Benjamin_Loison closed issue Benjamin_Loison/YouTube_captions_search_engine#26 2023-01-14 15:14:26 +01:00
Keep efficient search algorithm while keeping order (notably of the starting set)
6ce29051c0 Fix #26: Keep efficient search algorithm while keeping order (notably of the starting set)
Benjamin_Loison pushed to master at Benjamin_Loison/Synchronous_reactive_systems 2023-01-10 00:11:38 +01:00
b0545a2733 Correct some typos