Compare commits

...

42 Commits

Author SHA1 Message Date
Benex254
212f2af39c chore: bump version (v2.6.5) 2024-10-06 01:05:28 +03:00
Benex254
f7b2b4e0c9 feat: add serve command 2024-10-06 01:04:20 +03:00
Benedict Xavier
a747529279 Update README.md 2024-10-05 19:37:19 +03:00
Benex254
1dfdcc27ce chore: bump version (v2.6.4) 2024-10-05 12:33:32 +03:00
Benex254
3c03289453 fix: add git push to make_release 2024-10-05 12:33:23 +03:00
Benex254
06fd446a72 chore: bump version (v2.6.3) 2024-10-05 12:29:29 +03:00
Benex254
172d912d8b chore(release): improve the make release script to also stage changes after bumping version 2024-10-05 12:29:15 +03:00
Benex254
2396018607 feat: make script to automate releases 2024-10-05 12:19:03 +03:00
Benex254
a9be9779c5 feat(fa): improve fa script 2024-10-05 12:14:45 +03:00
Benex254
2f76b26a99 feat(fzf): add some bindings 2024-10-05 11:54:22 +03:00
Benex254
2fe5edf810 feat(cli): make all threads daemon threads 2024-10-05 11:47:52 +03:00
Benex254
d67ee6a779 feat(downloader): add progress hook option to be passed to yt-dlp 2024-10-05 11:47:30 +03:00
Benex254
e06ec5dbd4 feat(cli): make the image previews optional 2024-10-05 11:31:13 +03:00
Benex254
c1b24ba2aa feat(cli): save images with .png extenstion to enable easier viewing by external apps 2024-10-05 11:05:07 +03:00
Benex254
59e9cf9fd0 feat: improve previews 2024-10-05 10:12:14 +03:00
Benex254
58761f5b96 chore: bump version 2024-10-04 19:44:54 +03:00
Benex254
ac959da229 feat: renable bg downloading function 2024-10-04 19:42:53 +03:00
benex
bacc8c48ec fix: image previews not showing up on windows 2024-10-04 11:03:54 +03:00
Benex254
905a159428 chore: add a mapping for re:zero s3 in normalizer 2024-10-03 15:09:51 +03:00
Benex254
20f734cab2 feat: also compare synonymns 2024-10-03 15:09:14 +03:00
Benex254
7c2c644aef chore: bump version 2024-10-03 14:18:22 +03:00
Benex254
0efc92081a feat: use .get in normlizer 2024-10-03 14:18:05 +03:00
Benex254
fafeee2367 chore: bump version 2024-10-03 12:48:41 +03:00
Benex254
e03063cd76 feat: let configuration of providers be managed by AnimeProvider wrapper 2024-10-03 12:34:34 +03:00
Benex254
93b38b055f docs: update readme 2024-10-03 12:33:52 +03:00
Benex254
045635fb55 feat: update config.py 2024-10-03 12:33:40 +03:00
Benex254
de7f773e9e feat: make the threads non-daemon 2024-10-03 11:47:27 +03:00
Benex254
ef6a465bd2 fix: typing issue 2024-10-02 22:19:02 +03:00
Benex254
0c623af8a4 chore: update poetry lockfile 2024-10-02 21:57:17 +03:00
Benex254
0589f83998 chore: bump version 2024-10-02 21:56:50 +03:00
Benex254
e17608afd5 feat: add provider store 2024-10-02 21:33:41 +03:00
Benex254
b915654685 feat: make the requests cache allow multiple connections by switching to wal 2024-10-02 16:49:36 +03:00
Benex254
2ce9bf6c47 feat: use a more meaningful name for the request caching files 2024-09-30 13:20:14 +03:00
Benex254
3c22232432 feat: add option to delete the db file 2024-09-30 13:19:33 +03:00
Benex254
3474e9520c tests: pass custom env 2024-09-29 22:09:57 +03:00
Benex254
e9bacf4f9c fix: extra atexit callbacks 2024-09-29 21:31:58 +03:00
Benex254
ef422ed6fd fix: inability to reload provider dynamically when using cached sessions 2024-09-29 21:19:15 +03:00
Benex254
d0f5366908 feat: allow access of fastanime config from environment variables 2024-09-29 21:00:41 +03:00
Benex254
3557205feb chore: cleanup requests cacher 2024-09-29 20:41:55 +03:00
Benex254
ba4c41d888 feat: implement usage of the requests cacher 2024-09-29 20:40:14 +03:00
Benex254
1427a3193c feat: implement requests cacher for fastanime 2024-09-29 20:39:27 +03:00
Benex254
b5cee20e56 fix: episodes range generated by mini anilist 2024-09-28 10:31:33 +03:00
34 changed files with 1918 additions and 735 deletions

150
README.md
View File

@@ -8,12 +8,12 @@
Welcome to **FastAnime**, anime site experience from the terminal.
![fastanime-demo](https://github.com/user-attachments/assets/16e29f54-e9fa-48c7-b944-bfacb31ae1b5)
![fastanime](https://github.com/user-attachments/assets/9ab09f26-e4a8-4b70-a315-7def998cec63)
<details>
<summary><b>fzf mode</b></summary>
[fa_fzf_demo.webm](https://github.com/user-attachments/assets/b1fecf25-e358-4e8b-a144-bcb7947210cf)
[fastanime-fzf.webm](https://github.com/user-attachments/assets/90875a57-198b-4c78-98d5-10a459001edd)
</details>
@@ -35,7 +35,7 @@ Heavily inspired by [animdl](https://github.com/justfoolingaround/animdl), [jerr
<!--toc:start-->
- [FastAnime](#fastanime)
- [**FastAnime**](#fastanime)
- [Installation](#installation)
- [Installation using your favourite package manager](#installation-using-your-favourite-package-manager)
- [Using pipx](#using-pipx)
@@ -685,196 +685,74 @@ The app includes sensible defaults but can be customized extensively. Configurat
> `fastanime --icons --fzf --preview config --update`
> the above will set icons to true, use_fzf to true and preview to true in your config file
By default if a config file does not exist it will be auto created with comments to explain each and every option.
The default config:
```ini
#
# ███████╗░█████╗░░██████╗████████╗░█████╗░███╗░░██╗██╗███╗░░░███╗███████╗ ░█████╗░░█████╗░███╗░░██╗███████╗██╗░██████╗░
# ██╔════╝██╔══██╗██╔════╝╚══██╔══╝██╔══██╗████╗░██║██║████╗░████║██╔════╝ ██╔══██╗██╔══██╗████╗░██║██╔════╝██║██╔════╝░
# █████╗░░███████║╚█████╗░░░░██║░░░███████║██╔██╗██║██║██╔████╔██║█████╗░░ ██║░░╚═╝██║░░██║██╔██╗██║█████╗░░██║██║░░██╗░
# ██╔══╝░░██╔══██║░╚═══██╗░░░██║░░░██╔══██║██║╚████║██║██║╚██╔╝██║██╔══╝░░ ██║░░██╗██║░░██║██║╚████║██╔══╝░░██║██║░░╚██╗
# ██║░░░░░██║░░██║██████╔╝░░░██║░░░██║░░██║██║░╚███║██║██║░╚═╝░██║███████╗ ╚█████╔╝╚█████╔╝██║░╚███║██║░░░░░██║╚██████╔╝
# ╚═╝░░░░░╚═╝░░╚═╝╚═════╝░░░░╚═╝░░░╚═╝░░╚═╝╚═╝░░╚══╝╚═╝╚═╝░░░░░╚═╝╚══════╝ ░╚════╝░░╚════╝░╚═╝░░╚══╝╚═╝░░░░░╚═╝░╚═════╝░
#
[general]
# whether to show the icons in the tui [True/False]
# more like emojis
# by the way if you have any recommendations to which should be used where please
# don't hesitate to share your opinion
# cause it's a lot of work to look for the right one for each menu option
# be sure to also give the replacement emoji
icons = False
# the quality of the stream [1080,720,480,360]
# this option is usually only reliable when:
# provider=animepahe
# since it provides links that actually point to streams of different qualities
# while the rest just point to another link that can provide the anime from the same server
quality = 1080
# whether to normalize provider titles [True/False]
# basically takes the provider titles and finds the corresponding anilist title then changes the title to that
# useful for uniformity especially when downloading from different providers
# this also applies to episode titles
normalize_titles = True
# can be [allanime, animepahe, hianime]
# allanime is the most realible
# animepahe provides different links to streams of different quality so a quality can be selected reliably with --quality option
# hianime which is now hianime usually provides subs in different languuages and its servers are generally faster
provider = allanime
# Display language [english, romaji]
# this is passed to anilist directly and is used to set the language which the anime titles will be in
# when using the anilist interface
preferred_language = english
# Download directory
# where you will find your videos after downloading them with 'fastanime download' command
downloads_dir = ~/Videos/FastAnime
# whether to show a preview window when using fzf or rofi [True/False]
# the preview requires you have a commandline image viewer as documented in the README
# this is only when usinf fzf
# if you dont care about image previews it doesnt matter
# though its awesome
# try it and you will see
preview = False
# the time to seek when using ffmpegthumbnailer [-1 to 100]
# -1 means random and is the default
# ffmpegthumbnailer is used to generate previews and you can select at what time in the video to extract an image
# random makes things quite exciting cause you never no at what time it will extract the image from
ffmpegthumbnailer_seek_time = -1
# whether to use fzf as the interface for the anilist command and others. [True/False]
use_fzf = False
# whether to use rofi for the ui [True/False]
# it's more useful if you want to create a desktop entry
# which can be setup with 'fastanime config --desktop-entry'
# though if you want it to be your sole interface even when fastanime is run directly from the terminal
use_rofi = False
# rofi themes to use
# the values of this option is the path to the rofi config files to use
# i choose to split it into three since it gives the best look and feel
# you can refer to the rofi demo on github to see for your self
# by the way i recommend getting the rofi themes from this project;
rofi_theme =
rofi_theme_input =
rofi_theme_confirm =
# the duration in minutes a notification will stay in the screen
# used by notifier command
notification_duration = 2
# used when the provider gives subs of different languages
# currently its the case for:
# hianime
# the values for this option are the short names for countries
# regex is used to determine what you selected
sub_lang = eng
default_media_list_tracking = None
force_forward_tracking = True
cache_requests = True
use_persistent_provider_store = False
[stream]
# Auto continue from watch history [True/False]
# this will make fastanime to choose the episode that you last watched to completion
# and increment it by one
# and use that to auto select the episode you want to watch
continue_from_history = True
# which history to use [local/remote]
# local history means it will just use the watch history stored locally in your device
# the file that stores it is called watch_history.json and is stored next to your config file
# remote means it ignores the last episode stored locally and instead uses the one in your anilist anime list
# this config option is useful if you want to overwrite your local history or import history covered from another device or platform
# since remote history will take precendence over whats available locally
preferred_history = local
# Preferred language for anime [dub/sub]
translation_type = sub
# what server to use for a particular provider
# allanime: [dropbox, sharepoint, wetransfer, gogoanime, wixmp]
# animepahe: [kwik]
# hianime: [HD1, HD2, StreamSB, StreamTape]
# 'top' can also be used as a value for this option
# 'top' will cause fastanime to auto select the first server it sees
# this saves on resources and is faster since not all servers are being fetched
server = top
# Auto select next episode [True/False]
# this makes fastanime increment the current episode number
# then after using that value to fetch the next episode instead of prompting
# this option is useful for binging
auto_next = False
# Auto select the anime provider results with fuzzy find. [True/False]
# Note this won't always be correct
# this is because the providers sometime use non-standard names
# that are there own preference rather than the official names
# But 99% of the time will be accurate
# if this happens just turn of auto_select in the menus or from the commandline and manually select the correct anime title
# and then please open an issue at <> highlighting the normalized title and the title given by the provider for the anime you wished to watch
# or even better edit this file <> and open a pull request
auto_select = True
# whether to skip the opening and ending theme songs [True/False]
# NOTE: requires ani-skip to be in path
# for python-mpv users am planning to create this functionality n python without the use of an external script
# so its disabled for now
skip = False
# the maximum delta time in minutes after which the episode should be considered as completed
# used in the continue from time stamp
error = 3
episode_complete_at = 80
# whether to use python-mpv [True/False]
# to enable superior control over the player
# adding more options to it
# Enable this one and you will be wonder why you did not discover fastanime sooner
# Since you basically don't have to close the player window to go to the next or previous episode, switch servers, change translation type or
change to a given episode x
# so try it if you haven't already
# if you have any issues setting it up
# don't be afraid to ask
# especially on windows
# honestly it can be a pain to set it up there
# personally it took me quite sometime to figure it out
# this is because of how windows handles shared libraries
# so just ask when you find yourself stuck
# or just switch to arch linux
use_python_mpv = False
# force mpv window
# the default 'immediate' just makes mpv to open the window even if the video has not yet loaded
# done for asthetics
# passed directly to mpv so values are same
force_window = immediate
# the format of downloaded anime and trailer
# based on yt-dlp format and passed directly to it
# learn more by looking it up on their site
# only works for downloaded anime if:
# provider=allanime, server=gogoanime
# provider=allanime, server=wixmp
# provider=hianime
# this is because they provider a m3u8 file that contans multiple quality streams
format = best[height<=1080]/bestvideo[height<=1080]+bestaudio/best
# NOTE:
# if you have any trouble setting up your config
# please don't be afraid to ask in our discord
# plus if there are any errors, improvements or suggestions please tell us in the discord
# or help us by contributing
# we appreciate all the help we can get
# since we may not always have the time to immediately implement the changes
#
# HOPE YOU ENJOY FASTANIME AND BE SURE TO STAR THE PROJECT ON GITHUB
#
player = mpv
```
## Contributing

5
fa
View File

@@ -1,4 +1,3 @@
#!/usr/bin/env sh
# exec "${PYTHON:-python3}" -Werror -Xdev -m "$(dirname "$(realpath "$0")")/fastanime" "$@"
cd "$(dirname "$(realpath "$0")")" || exit 1
exec python -m fastanime "$@"
CLI_DIR="$(dirname "$(realpath "$0")")"
exec python -m "$CLI_DIR/fastanime" "$@"

View File

@@ -1,10 +1,8 @@
"""An abstraction over all providers offering added features with a simple and well typed api
[TODO:description]
"""
"""An abstraction over all providers offering added features with a simple and well typed api"""
import importlib
import logging
import os
from typing import TYPE_CHECKING
from .libs.anime_provider import anime_sources
@@ -32,19 +30,36 @@ class AnimeProvider:
PROVIDERS = list(anime_sources.keys())
provider = PROVIDERS[0]
def __init__(self, provider, dynamic=False, retries=0) -> None:
def __init__(
self,
provider,
cache_requests=os.environ.get("FASTANIME_CACHE_REQUESTS", "false"),
use_persistent_provider_store=os.environ.get(
"FASTANIME_USE_PERSISTENT_PROVIDER_STORE", "false"
),
dynamic=False,
retries=0,
) -> None:
self.provider = provider
self.dynamic = dynamic
self.retries = retries
self.cache_requests = cache_requests
self.use_persistent_provider_store = use_persistent_provider_store
self.lazyload_provider(self.provider)
def lazyload_provider(self, provider):
"""updates the current provider being used"""
try:
self.anime_provider.session.kill_connection_to_db()
except Exception:
pass
_, anime_provider_cls_name = anime_sources[provider].split(".", 1)
package = f"fastanime.libs.anime_provider.{provider}"
provider_api = importlib.import_module(".api", package)
anime_provider = getattr(provider_api, anime_provider_cls_name)
self.anime_provider = anime_provider()
self.anime_provider = anime_provider(
self.cache_requests, self.use_persistent_provider_store
)
def search_for_anime(
self,
@@ -93,7 +108,6 @@ class AnimeProvider:
def get_episode_streams(
self,
anime_id,
anime_title,
episode: str,
translation_type: str,
) -> "Iterator[Server] | None":
@@ -110,6 +124,6 @@ class AnimeProvider:
"""
anime_provider = self.anime_provider
results = anime_provider.get_episode_streams(
anime_id, anime_title, episode, translation_type
anime_id, episode, translation_type
)
return results

View File

@@ -1,13 +1,16 @@
import re
from datetime import datetime
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from ..libs.anilist.types import AnilistDateObject, AnilistMediaNextAiringEpisode
COMMA_REGEX = re.compile(r"([0-9]{3})(?=\d)")
# TODO: Add formating options for the final date
def format_anilist_date_object(anilist_date_object: "AnilistDateObject"):
if anilist_date_object:
if anilist_date_object and anilist_date_object["day"]:
return f"{anilist_date_object['day']}/{anilist_date_object['month']}/{anilist_date_object['year']}"
else:
return "Unknown"
@@ -27,6 +30,12 @@ def format_list_data_with_comma(data: list | None):
return "None"
def format_number_with_commas(number: int | None):
if not number:
return "0"
return COMMA_REGEX.sub(lambda match: f"{match.group(1)},", str(number)[::-1])[::-1]
def extract_next_airing_episode(airing_episode: "AnilistMediaNextAiringEpisode"):
if airing_episode:
return f"{airing_episode['episode']} on {format_anilist_timestamp(airing_episode['airingAt'])}"

View File

@@ -9,6 +9,7 @@ anime_normalizer_raw = {
"Magia Record: Mahou Shoujo Madoka☆Magica Gaiden (TV)": "Mahou Shoujo Madoka☆Magica",
"Dungeon ni Deai o Motomeru no wa Machigatte Iru Darouka": "Dungeon ni Deai wo Motomeru no wa Machigatteiru Darou ka",
'Hazurewaku no "Joutai Ijou Skill" de Saikyou ni Natta Ore ga Subete wo Juurin suru made': "Hazure Waku no [Joutai Ijou Skill] de Saikyou ni Natta Ore ga Subete wo Juurin Suru made",
"Re:Zero kara Hajimeru Isekai Seikatsu Season 3": "Re:Zero kara Hajimeru Isekai Seikatsu 3rd Season",
},
"hianime": {"My Star": "Oshi no Ko"},
"animepahe": {"Azumanga Daiou The Animation": "Azumanga Daioh"},
@@ -20,7 +21,7 @@ def get_anime_normalizer():
"""Used because there are different providers"""
import os
current_provider = os.environ["CURRENT_FASTANIME_PROVIDER"]
current_provider = os.environ.get("FASTANIME_PROVIDER", "allanime")
return anime_normalizer_raw[current_provider]

View File

@@ -16,6 +16,7 @@ logger = logging.getLogger(__name__)
class YtDLPDownloader:
downloads_queue = Queue()
_thread = None
def _worker(self):
while True:
@@ -26,11 +27,6 @@ class YtDLPDownloader:
logger.error(f"Something went wrong {e}")
self.downloads_queue.task_done()
def __init__(self):
self._thread = Thread(target=self._worker)
self._thread.daemon = True
self._thread.start()
def _download_file(
self,
url: str,
@@ -38,6 +34,7 @@ class YtDLPDownloader:
episode_title: str,
download_dir: str,
silent: bool,
progress_hooks=[],
vid_format: str = "best",
force_unknown_ext=False,
verbose=False,
@@ -86,6 +83,7 @@ class YtDLPDownloader:
"verbose": verbose,
"format": vid_format,
"compat_opts": ("allow-unsafe-ext",) if force_unknown_ext else tuple(),
"progress_hooks": progress_hooks,
}
urls = [url]
if sub:
@@ -174,8 +172,15 @@ class YtDLPDownloader:
except Exception as e:
print(f"[red bold]An error[/] occurred: {e}")
# WARN: May remove this legacy functionality
def download_file(self, url: str, title, silent=True):
def download_file(
self,
url: str,
anime_title: str,
episode_title: str,
download_dir: str,
silent: bool = True,
**kwargs,
):
"""A helper that just does things in the background
Args:
@@ -183,7 +188,17 @@ class YtDLPDownloader:
silent ([TODO:parameter]): [TODO:description]
url: [TODO:description]
"""
self.downloads_queue.put((self._download_file, (url, title, silent)))
if not self._thread:
self._thread = Thread(target=self._worker)
self._thread.daemon = True
self._thread.start()
self.downloads_queue.put(
(
self._download_file,
(url, anime_title, episode_title, download_dir, silent),
)
)
downloader = YtDLPDownloader()

View File

@@ -37,6 +37,10 @@ def anime_title_percentage_match(
title_a = str(anime["title"]["romaji"])
title_b = str(anime["title"]["english"])
percentage_ratio = max(
*[
fuzz.ratio(title.lower(), possible_user_requested_anime_title.lower())
for title in anime["synonyms"]
],
fuzz.ratio(title_a.lower(), possible_user_requested_anime_title.lower()),
fuzz.ratio(title_b.lower(), possible_user_requested_anime_title.lower()),
)

View File

@@ -2,11 +2,11 @@ import sys
if sys.version_info < (3, 10):
raise ImportError(
"You are using an unsupported version of Python. Only Python versions 3.8 and above are supported by yt-dlp"
"You are using an unsupported version of Python. Only Python versions 3.10 and above are supported by FastAnime"
) # noqa: F541
__version__ = "v2.5.6"
__version__ = "v2.6.5"
APP_NAME = "FastAnime"
AUTHOR = "Benex254"

25
fastanime/api/__init__.py Normal file
View File

@@ -0,0 +1,25 @@
from typing import Literal
from fastapi import FastAPI
from ..AnimeProvider import AnimeProvider
app = FastAPI()
anime_provider = AnimeProvider("allanime", "true", "true")
@app.get("/search")
def search_for_anime(title: str, translation_type: Literal["dub", "sub"] = "sub"):
return anime_provider.search_for_anime(title, translation_type)
@app.get("/anime/{anime_id}")
def get_anime(anime_id: str):
return anime_provider.get_anime(anime_id)
@app.get("/anime/{anime_id}/watch")
def get_episode_streams(
anime_id: str, episode: str, translation_type: Literal["sub", "dub"]
):
return anime_provider.get_episode_streams(anime_id, episode, translation_type)

View File

@@ -16,6 +16,7 @@ commands = {
"completions": "completions.completions",
"update": "update.update",
"grab": "grab.grab",
"serve": "serve.serve",
}
@@ -253,10 +254,7 @@ def run_cli(
if sync_play:
ctx.obj.sync_play = sync_play
if provider:
import os
ctx.obj.provider = provider
os.environ["CURRENT_FASTANIME_PROVIDER"] = provider
if server:
ctx.obj.server = server
if format:
@@ -330,3 +328,4 @@ def run_cli(
if rofi_theme_confirm:
ctx.obj.rofi_theme_confirm = rofi_theme_confirm
Rofi.rofi_theme_confirm = rofi_theme_confirm
ctx.obj.set_fastanime_config_environs()

View File

@@ -284,7 +284,7 @@ def download(
with Progress() as progress:
progress.add_task("Fetching Episode Streams...", total=None)
streams = anime_provider.get_episode_streams(
anime["id"], anime["title"], episode, config.translation_type
anime["id"], episode, config.translation_type
)
if not streams:
print("No streams skipping")
@@ -361,9 +361,9 @@ def download(
episode_title,
download_dir,
silent,
config.format,
force_unknown_ext,
verbose,
vid_format=config.format,
force_unknown_ext=force_unknown_ext,
verbose=verbose,
headers=provider_headers,
sub=subtitles[0]["url"] if subtitles else "",
merge=merge,

View File

@@ -217,7 +217,7 @@ def grab(
if episode not in episodes:
continue
streams = anime_provider.get_episode_streams(
anime["id"], anime["title"], episode, config.translation_type
anime["id"], episode, config.translation_type
)
if not streams:
continue

View File

@@ -283,7 +283,7 @@ def search(config: "Config", anime_titles: str, episode_range: str):
with Progress() as progress:
progress.add_task("Fetching Episode Streams...", total=None)
streams = anime_provider.get_episode_streams(
anime["id"], anime["title"], episode, config.translation_type
anime["id"], episode, config.translation_type
)
if not streams:
print("Failed to get streams")

View File

@@ -0,0 +1,31 @@
import click
@click.command(
help="Command that automates the starting of the builtin fastanime server",
epilog="""
\b
\b\bExamples:
# default
fastanime serve
# specify host and port
fastanime serve --host 127.0.0.1 --port 8080
""",
)
@click.option("--host", "-H", help="Specify the host to run the server on")
@click.option("--port", "-p", help="Check for the latest release", type=int)
def serve(host, port):
import os
import sys
from ...constants import APP_DIR
args = ["python", "-m", "fastapi", "run"]
if host:
args.extend(["--host", host])
if port:
args.extend(["--port", port])
args.append(os.path.join(APP_DIR, "api"))
os.execv(sys.executable, args)

View File

@@ -27,37 +27,40 @@ class Config(object):
)
anime_provider: "AnimeProvider"
user_data = {"watch_history": {}, "animelist": [], "user": {}}
default_options = {
"quality": "1080",
default_config = {
"auto_next": "False",
"auto_select": "True",
"sort_by": "search match",
"downloads_dir": USER_VIDEOS_DIR,
"translation_type": "sub",
"server": "top",
"cache_requests": "true",
"continue_from_history": "True",
"preferred_history": "local",
"use_python_mpv": "false",
"force_window": "immediate",
"preferred_language": "english",
"use_fzf": "False",
"preview": "False",
"format": "best[height<=1080]/bestvideo[height<=1080]+bestaudio/best",
"provider": "allanime",
"icons": "false",
"notification_duration": "2",
"skip": "false",
"use_rofi": "false",
"rofi_theme": "",
"rofi_theme_input": "",
"rofi_theme_confirm": "",
"ffmpegthumnailer_seek_time": "-1",
"sub_lang": "eng",
"normalize_titles": "true",
"player": "mpv",
"episode_complete_at": "80",
"force_forward_tracking": "true",
"default_media_list_tracking": "None",
"downloads_dir": USER_VIDEOS_DIR,
"episode_complete_at": "80",
"ffmpegthumbnailer_seek_time": "-1",
"force_forward_tracking": "true",
"force_window": "immediate",
"format": "best[height<=1080]/bestvideo[height<=1080]+bestaudio/best",
"icons": "false",
"image_previews": "true",
"normalize_titles": "true",
"notification_duration": "2",
"player": "mpv",
"preferred_history": "local",
"preferred_language": "english",
"preview": "False",
"provider": "allanime",
"quality": "1080",
"rofi_theme": "",
"rofi_theme_confirm": "",
"rofi_theme_input": "",
"server": "top",
"skip": "false",
"sort_by": "search match",
"sub_lang": "eng",
"translation_type": "sub",
"use_fzf": "False",
"use_persistent_provider_store": "false",
"use_python_mpv": "false",
"use_rofi": "false",
}
def __init__(self) -> None:
@@ -65,7 +68,7 @@ class Config(object):
self.load_config()
def load_config(self):
self.configparser = ConfigParser(self.default_options)
self.configparser = ConfigParser(self.default_config)
self.configparser.add_section("stream")
self.configparser.add_section("general")
self.configparser.add_section("anilist")
@@ -74,48 +77,60 @@ class Config(object):
if os.path.exists(USER_CONFIG_PATH):
self.configparser.read(USER_CONFIG_PATH, encoding="utf-8")
self.downloads_dir = self.get_downloads_dir()
self.sub_lang = self.get_sub_lang()
self.provider = self.get_provider()
self.use_fzf = self.get_use_fzf()
self.use_rofi = self.get_use_rofi()
self.skip = self.get_skip()
self.icons = self.get_icons()
self.preview = self.get_preview()
self.translation_type = self.get_translation_type()
self.sort_by = self.get_sort_by()
self.continue_from_history = self.get_continue_from_history()
self.auto_next = self.get_auto_next()
self.normalize_titles = self.get_normalize_titles()
self.auto_select = self.get_auto_select()
self.use_python_mpv = self.get_use_mpv_mod()
self.quality = self.get_quality()
self.notification_duration = self.get_notification_duration()
self.episode_complete_at = self.get_episode_complete_at()
self.cache_requests = self.get_cache_requests()
self.continue_from_history = self.get_continue_from_history()
self.default_media_list_tracking = self.get_default_media_list_tracking()
self.force_forward_tracking = self.get_force_forward_tracking()
self.server = self.get_server()
self.format = self.get_format()
self.player = self.get_player()
self.force_window = self.get_force_window()
self.preferred_language = self.get_preferred_language()
self.preferred_history = self.get_preferred_history()
self.rofi_theme = self.get_rofi_theme()
Rofi.rofi_theme = self.rofi_theme
self.rofi_theme_input = self.get_rofi_theme_input()
Rofi.rofi_theme_input = self.rofi_theme_input
self.rofi_theme_confirm = self.get_rofi_theme_confirm()
Rofi.rofi_theme_confirm = self.rofi_theme_confirm
self.downloads_dir = self.get_downloads_dir()
self.episode_complete_at = self.get_episode_complete_at()
self.ffmpegthumbnailer_seek_time = self.get_ffmpegthumnailer_seek_time()
self.force_forward_tracking = self.get_force_forward_tracking()
self.force_window = self.get_force_window()
self.format = self.get_format()
self.icons = self.get_icons()
self.image_previews = self.get_image_previews()
self.normalize_titles = self.get_normalize_titles()
self.notification_duration = self.get_notification_duration()
self.player = self.get_player()
self.preferred_history = self.get_preferred_history()
self.preferred_language = self.get_preferred_language()
self.preview = self.get_preview()
self.provider = self.get_provider()
self.quality = self.get_quality()
self.rofi_theme_confirm = self.get_rofi_theme_confirm()
self.rofi_theme_input = self.get_rofi_theme_input()
self.rofi_theme = self.get_rofi_theme()
Rofi.rofi_theme_confirm = self.rofi_theme_confirm
Rofi.rofi_theme_input = self.rofi_theme_input
Rofi.rofi_theme = self.rofi_theme
self.server = self.get_server()
self.skip = self.get_skip()
self.sort_by = self.get_sort_by()
self.sub_lang = self.get_sub_lang()
self.translation_type = self.get_translation_type()
self.use_fzf = self.get_use_fzf()
self.use_python_mpv = self.get_use_mpv_mod()
self.use_rofi = self.get_use_rofi()
self.use_persistent_provider_store = self.get_use_persistent_provider_store()
# ---- setup user data ------
self.anime_list: list = self.user_data.get("animelist", [])
self.user: dict = self.user_data.get("user", {})
os.environ["CURRENT_FASTANIME_PROVIDER"] = self.provider
if not os.path.exists(USER_CONFIG_PATH):
with open(USER_CONFIG_PATH, "w", encoding="utf-8") as config:
config.write(self.__repr__())
def set_fastanime_config_environs(self):
current_config = []
for key in self.default_config:
current_config.append((f"FASTANIME_{key.upper()}", str(getattr(self, key))))
os.environ.update(current_config)
def update_user(self, user):
self.user = user
self.user_data["user"] = user
@@ -170,7 +185,7 @@ class Config(object):
return self.configparser.get("general", "provider")
def get_ffmpegthumnailer_seek_time(self):
return self.configparser.getint("general", "ffmpegthumnailer_seek_time")
return self.configparser.getint("general", "ffmpegthumbnailer_seek_time")
def get_preferred_language(self):
return self.configparser.get("general", "preferred_language")
@@ -184,12 +199,18 @@ class Config(object):
def get_icons(self):
return self.configparser.getboolean("general", "icons")
def get_image_previews(self):
return self.configparser.getboolean("general", "image_previews")
def get_preview(self):
return self.configparser.getboolean("general", "preview")
def get_use_fzf(self):
return self.configparser.getboolean("general", "use_fzf")
def get_use_persistent_provider_store(self):
return self.configparser.getboolean("general", "use_persistent_provider_store")
# rofi conifiguration
def get_use_rofi(self):
return self.configparser.getboolean("general", "use_rofi")
@@ -206,6 +227,9 @@ class Config(object):
def get_force_forward_tracking(self):
return self.configparser.getboolean("general", "force_forward_tracking")
def get_cache_requests(self):
return self.configparser.getboolean("general", "cache_requests")
def get_default_media_list_tracking(self):
return self.configparser.get("general", "default_media_list_tracking")
@@ -318,6 +342,9 @@ downloads_dir = {self.downloads_dir}
# try it and you will see
preview = {self.preview}
# whether to show images in the preview [true/false]
image_previews = {self.image_previews}
# the time to seek when using ffmpegthumbnailer [-1 to 100]
# -1 means random and is the default
# ffmpegthumbnailer is used to generate previews and you can select at what time in the video to extract an image
@@ -366,6 +393,21 @@ default_media_list_tracking = {self.default_media_list_tracking}
# this affects only your anilist anime list
force_forward_tracking = {self.force_forward_tracking}
# whether to cache requests [true/false]
# this makes the experience better and more faster
# as data need not always be fetched from web server
# and instead can be gotten locally
# from the cached_requests_db
cache_requests = {self.cache_requests}
# whether to use a persistent store (basically a sqlitedb) for storing some data the provider requires
# to enable a seamless experience [true/false]
# this option exists primarily because i think it may help in the optimization
# of fastanime as a library in a website project
# for now i don't recommend changing it
# leave it as is
use_persistent_provider_store = {self.use_persistent_provider_store}
[stream]
# Auto continue from watch history [True/False]
@@ -426,7 +468,9 @@ episode_complete_at = {self.episode_complete_at}
# to enable superior control over the player
# adding more options to it
# Enable this one and you will be wonder why you did not discover fastanime sooner
# Since you basically don't have to close the player window to go to the next or previous episode, switch servers, change translation type or change to a given episode x
# Since you basically don't have to close the player window
# to go to the next or previous episode, switch servers,
# change translation type or change to a given episode x
# so try it if you haven't already
# if you have any issues setting it up
# don't be afraid to ask

View File

@@ -388,7 +388,6 @@ def provider_anime_episode_servers_menu(
progress.add_task("Fetching Episode Streams...", total=None)
episode_streams_generator = anime_provider.get_episode_streams(
provider_anime["id"],
provider_anime["title"],
current_episode_number,
translation_type,
)
@@ -1421,7 +1420,7 @@ def anilist_results_menu(
choices = []
for title in anime_data.keys():
icon_path = os.path.join(IMAGES_CACHE_DIR, title)
choices.append(f"{title}\0icon\x1f{icon_path}")
choices.append(f"{title}\0icon\x1f{icon_path}.png")
choices.append("Back")
selected_anime_title = Rofi.run_with_icons(choices, "Select Anime")
else:

View File

@@ -9,7 +9,7 @@ from threading import Thread
import requests
from yt_dlp.utils import clean_html, sanitize_filename
from ...constants import APP_CACHE_DIR
from ...constants import APP_CACHE_DIR, S_PLATFORM
from ...libs.anilist.types import AnilistBaseMediaDataSchema
from ...Utility import anilist_data_helper
from ..utils.scripts import fzf_preview
@@ -46,7 +46,9 @@ def aniskip(mal_id: int, episode: str):
# NOTE: May change this to a temp dir but there were issues so later
WORKING_DIR = APP_CACHE_DIR # tempfile.gettempdir()
HEADER_COLOR = 215, 0, 95
SEPARATOR_COLOR = 208, 208, 208
SINGLE_QUOTE = "'"
IMAGES_CACHE_DIR = os.path.join(WORKING_DIR, "images")
if not os.path.exists(IMAGES_CACHE_DIR):
os.mkdir(IMAGES_CACHE_DIR)
@@ -63,7 +65,7 @@ def save_image_from_url(url: str, file_name: str):
file_name: filename to use
"""
image = requests.get(url)
with open(f"{IMAGES_CACHE_DIR}/{file_name}", "wb") as f:
with open(f"{IMAGES_CACHE_DIR}/{file_name}.png", "wb") as f:
f.write(image.content)
@@ -91,18 +93,16 @@ def write_search_results(
workers:number of threads to use defaults to as many as possible
"""
# NOTE: Will probably make this a configuraable option
HEADER_COLOR = 215, 0, 95
SEPARATOR_COLOR = 208, 208, 208
SEPARATOR_WIDTH = 30
# use concurency to download and write as fast as possible
with concurrent.futures.ThreadPoolExecutor(max_workers=workers) as executor:
future_to_task = {}
for anime, title in zip(anilist_results, titles):
# actual image url
image_url = anime["coverImage"]["large"]
future_to_task[executor.submit(save_image_from_url, image_url, title)] = (
image_url
)
if os.environ.get("FASTANIME_IMAGE_PREVIEWS", "true").lower() == "true":
image_url = anime["coverImage"]["large"]
future_to_task[
executor.submit(save_image_from_url, image_url, title)
] = image_url
mediaListName = "Not in any of your lists"
progress = "UNKNOWN"
@@ -111,28 +111,57 @@ def write_search_results(
progress = anime_list["progress"]
# handle the text data
template = f"""
{get_true_fg("-"*SEPARATOR_WIDTH,*SEPARATOR_COLOR,bold=False)}
{get_true_fg('Title(jp):',*HEADER_COLOR)} {anime['title']['romaji']}
{get_true_fg('Title(eng):',*HEADER_COLOR)} {anime['title']['english']}
{get_true_fg('Popularity:',*HEADER_COLOR)} {anime['popularity']}
{get_true_fg('Favourites:',*HEADER_COLOR)} {anime['favourites']}
{get_true_fg('Status:',*HEADER_COLOR)} {anime['status']}
{get_true_fg('Episodes:',*HEADER_COLOR)} {anime['episodes']}
{get_true_fg('Genres:',*HEADER_COLOR)} {anilist_data_helper.format_list_data_with_comma(anime['genres'])}
{get_true_fg('Next Episode:',*HEADER_COLOR)} {anilist_data_helper.extract_next_airing_episode(anime['nextAiringEpisode'])}
{get_true_fg('Start Date:',*HEADER_COLOR)} {anilist_data_helper.format_anilist_date_object(anime['startDate'])}
{get_true_fg('End Date:',*HEADER_COLOR)} {anilist_data_helper.format_anilist_date_object(anime['endDate'])}
{get_true_fg("-"*SEPARATOR_WIDTH,*SEPARATOR_COLOR,bold=False)}
{get_true_fg('Media List:',*HEADER_COLOR)} {mediaListName}
{get_true_fg('Progress:',*HEADER_COLOR)} {progress}
{get_true_fg("-"*SEPARATOR_WIDTH,*SEPARATOR_COLOR,bold=False)}
{get_true_fg('Description:',*HEADER_COLOR)}
ll=2
while [ $ll -le $FZF_PREVIEW_COLUMNS ];do
echo -n -e "{get_true_fg("",*SEPARATOR_COLOR,bold=False)}"
((ll++))
done
echo
echo "{get_true_fg('Title(jp):',*HEADER_COLOR)} {(anime['title']['romaji'] or "").replace('"',SINGLE_QUOTE)}"
echo "{get_true_fg('Title(eng):',*HEADER_COLOR)} {(anime['title']['english'] or "").replace('"',SINGLE_QUOTE)}"
ll=2
while [ $ll -le $FZF_PREVIEW_COLUMNS ];do
echo -n -e "{get_true_fg("",*SEPARATOR_COLOR,bold=False)}"
((ll++))
done
echo
echo "{get_true_fg('Popularity:',*HEADER_COLOR)} {anilist_data_helper.format_number_with_commas(anime['popularity'])}"
echo "{get_true_fg('Favourites:',*HEADER_COLOR)} {anilist_data_helper.format_number_with_commas(anime['favourites'])}"
echo "{get_true_fg('Status:',*HEADER_COLOR)} {str(anime['status']).replace('"',SINGLE_QUOTE)}"
echo "{get_true_fg('Next Episode:',*HEADER_COLOR)} {anilist_data_helper.extract_next_airing_episode(anime['nextAiringEpisode']).replace('"',SINGLE_QUOTE)}"
echo "{get_true_fg('Genres:',*HEADER_COLOR)} {anilist_data_helper.format_list_data_with_comma(anime['genres']).replace('"',SINGLE_QUOTE)}"
ll=2
while [ $ll -le $FZF_PREVIEW_COLUMNS ];do
echo -n -e "{get_true_fg("",*SEPARATOR_COLOR,bold=False)}"
((ll++))
done
echo
echo "{get_true_fg('Episodes:',*HEADER_COLOR)} {(anime['episodes']) or 'UNKNOWN'}"
echo "{get_true_fg('Start Date:',*HEADER_COLOR)} {anilist_data_helper.format_anilist_date_object(anime['startDate']).replace('"',SINGLE_QUOTE)}"
echo "{get_true_fg('End Date:',*HEADER_COLOR)} {anilist_data_helper.format_anilist_date_object(anime['endDate']).replace('"',SINGLE_QUOTE)}"
ll=2
while [ $ll -le $FZF_PREVIEW_COLUMNS ];do
echo -n -e "{get_true_fg("",*SEPARATOR_COLOR,bold=False)}"
((ll++))
done
echo
echo "{get_true_fg('Media List:',*HEADER_COLOR)} {mediaListName.replace('"',SINGLE_QUOTE)}"
echo "{get_true_fg('Progress:',*HEADER_COLOR)} {progress}"
ll=2
while [ $ll -le $FZF_PREVIEW_COLUMNS ];do
echo -n -e "{get_true_fg("",*SEPARATOR_COLOR,bold=False)}"
((ll++))
done
echo
# echo "{get_true_fg('Description:',*HEADER_COLOR).replace('"',SINGLE_QUOTE)}"
"""
template = textwrap.dedent(template)
template = f"""
{template}
echo "
{textwrap.fill(clean_html(
str(anime['description'])), width=45)}
(anime['description']) or "").replace('"',SINGLE_QUOTE), width=45)}
"
"""
future_to_task[executor.submit(save_info_from_str, template, title)] = title
@@ -212,8 +241,8 @@ def get_fzf_manga_preview(manga_results, workers=None, wait=False):
background_worker = Thread(
target=_worker,
)
# ensure images and info exists
background_worker.daemon = True
# ensure images and info exists
background_worker.start()
# the preview script is in bash so making sure fzf doesnt use any other shell lang to process the preview script
@@ -270,8 +299,13 @@ def get_fzf_episode_preview(
] = image_url
template = textwrap.dedent(
f"""
{get_true_fg('Anime Title:',*HEADER_COLOR)} {anilist_result['title']['romaji'] or anilist_result['title']['english']}
{get_true_fg('Episode Title:',*HEADER_COLOR)} {episode_title}
ll=2
while [ $ll -le $FZF_PREVIEW_COLUMNS ];do
echo -n -e "{get_true_fg("",*SEPARATOR_COLOR,bold=False)}"
((ll++))
done
echo "{get_true_fg('Anime Title:',*HEADER_COLOR)} {(anilist_result['title']['romaji'] or anilist_result['title']['english']).replace('"',SINGLE_QUOTE)}"
echo "{get_true_fg('Episode Title:',*HEADER_COLOR)} {str(episode_title).replace('"',SINGLE_QUOTE)}"
"""
)
future_to_url[
@@ -289,27 +323,61 @@ def get_fzf_episode_preview(
background_worker = Thread(
target=_worker,
)
# ensure images and info exists
background_worker.daemon = True
# ensure images and info exists
background_worker.start()
# the preview script is in bash so making sure fzf doesnt use any other shell lang to process the preview script
os.environ["SHELL"] = shutil.which("bash") or "bash"
preview = """
%s
if [ -s %s/{} ]; then fzf-preview %s/{}
else echo Loading...
fi
if [ -s %s/{} ]; then cat %s/{}
else echo Loading...
fi
""" % (
fzf_preview,
IMAGES_CACHE_DIR,
IMAGES_CACHE_DIR,
ANIME_INFO_CACHE_DIR,
ANIME_INFO_CACHE_DIR,
)
if S_PLATFORM == "win32":
preview = """
%s
title={}
show_image_previews="%s"
dim=${FZF_PREVIEW_COLUMNS}x${FZF_PREVIEW_LINES}
if [ $show_image_previews = "true" ];then
if [ -s "%s\\\\\\${title}.png" ]; then
if command -v "chafa">/dev/null;then
chafa -s $dim "%s\\\\\\${title}.png"
else
echo please install chafa to enjoy image previews
fi
echo
else
echo Loading...
fi
fi
if [ -s "%s\\\\\\$title" ]; then source "%s\\\\\\$title"
else echo Loading...
fi
""" % (
fzf_preview,
os.environ.get("FASTANIME_IMAGE_PREVIEWS", "true").lower(),
IMAGES_CACHE_DIR.replace("\\", "\\\\\\"),
IMAGES_CACHE_DIR.replace("\\", "\\\\\\"),
ANIME_INFO_CACHE_DIR.replace("\\", "\\\\\\"),
ANIME_INFO_CACHE_DIR.replace("\\", "\\\\\\"),
)
else:
preview = """
%s
show_image_previews="%s"
if [ $show_image_previews = "true" ];then
if [ -s %s/{} ]; then fzf-preview %s/{}
else echo Loading...
fi
fi
if [ -s %s/{} ]; then source %s/{}
else echo Loading...
fi
""" % (
fzf_preview,
os.environ.get("FASTANIME_IMAGE_PREVIEWS", "true").lower(),
IMAGES_CACHE_DIR,
IMAGES_CACHE_DIR,
ANIME_INFO_CACHE_DIR,
ANIME_INFO_CACHE_DIR,
)
if wait:
background_worker.join()
return preview
@@ -329,7 +397,7 @@ def get_fzf_anime_preview(
THe fzf preview script to use
"""
# ensure images and info exists
from ...constants import S_PLATFORM
background_worker = Thread(
target=write_search_results, args=(anilist_results, titles)
)
@@ -342,34 +410,47 @@ def get_fzf_anime_preview(
preview = """
%s
title={}
show_image_previews="%s"
dim=${FZF_PREVIEW_COLUMNS}x${FZF_PREVIEW_LINES}
if [ -s "%s\\\\\\$title" ]; then
if command -v chafa >/dev/null;then
chafa -f kitty -s $dim "%s\\\\\\$title"
if [ $show_image_previews = "true" ];then
if [ -s "%s\\\\\\${title}.png" ]; then
if command -v "chafa">/dev/null;then
chafa -s $dim "%s\\\\\\${title}.png"
else
echo please install chafa to enjoy image previews
fi
echo
else
echo Loading...
fi
fi
else echo Loading...
fi
if [ -s "%s\\\\\\$title" ]; then cat "%s\\\\\\$title"
else echo Loading...
if [ -s "%s\\\\\\$title" ]; then source "%s\\\\\\$title"
else echo Loading...
fi
""" % (
fzf_preview,
IMAGES_CACHE_DIR.replace("\\","\\\\\\"),
IMAGES_CACHE_DIR.replace("\\","\\\\\\"),
ANIME_INFO_CACHE_DIR.replace("\\","\\\\\\"),
ANIME_INFO_CACHE_DIR.replace("\\","\\\\\\"),
os.environ.get("FASTANIME_IMAGE_PREVIEWS", "true").lower(),
IMAGES_CACHE_DIR.replace("\\", "\\\\\\"),
IMAGES_CACHE_DIR.replace("\\", "\\\\\\"),
ANIME_INFO_CACHE_DIR.replace("\\", "\\\\\\"),
ANIME_INFO_CACHE_DIR.replace("\\", "\\\\\\"),
)
else:
preview = """
%s
if [ -s %s/{} ]; then fzf-preview %s/{}
else echo Loading...
title={}
show_image_previews="%s"
if [ $show_image_previews = "true" ];then
if [ -s "%s/${title}.png" ]; then fzf-preview "%s/${title}.png"
else echo Loading...
fi
fi
if [ -s %s/{} ]; then cat %s/{}
if [ -s "%s/$title" ]; then source "%s/$title"
else echo Loading...
fi
""" % (
fzf_preview,
os.environ.get("FASTANIME_IMAGE_PREVIEWS", "true").lower(),
IMAGES_CACHE_DIR,
IMAGES_CACHE_DIR,
ANIME_INFO_CACHE_DIR,

View File

@@ -121,7 +121,6 @@ class MpvPlayer(object):
# get them juicy streams
episode_streams = anime_provider.get_episode_streams(
provider_anime["id"],
provider_anime["title"],
current_episode_number,
translation_type,
)

View File

@@ -225,6 +225,7 @@ query ($userId: Int, $status: MediaListStatus, $type: MediaType) {
averageScore
episodes
genres
synonyms
studios {
nodes {
name
@@ -369,6 +370,7 @@ query($query:String,%s){
averageScore
episodes
genres
synonyms
studios {
nodes {
name
@@ -428,6 +430,7 @@ query ($type: MediaType) {
favourites
averageScore
genres
synonyms
episodes
description
studios {
@@ -503,6 +506,7 @@ query ($type: MediaType) {
episodes
description
genres
synonyms
studios {
nodes {
name
@@ -566,6 +570,7 @@ query ($type: MediaType) {
averageScore
description
genres
synonyms
studios {
nodes {
name
@@ -624,6 +629,7 @@ query ($type: MediaType) {
description
episodes
genres
synonyms
mediaListEntry {
status
id
@@ -698,6 +704,7 @@ query ($type: MediaType) {
averageScore
description
genres
synonyms
episodes
studios {
nodes {
@@ -759,6 +766,7 @@ query ($type: MediaType) {
id
}
genres
synonyms
averageScore
popularity
streamingEpisodes {
@@ -862,6 +870,7 @@ query ($id: Int, $type: MediaType) {
id
}
genres
synonyms
averageScore
popularity
streamingEpisodes {
@@ -954,6 +963,7 @@ query ($page: Int, $type: MediaType) {
favourites
averageScore
genres
synonyms
episodes
description
studios {

View File

@@ -26,6 +26,7 @@ class AllAnimeAPI(AnimeProvider):
Provides a fast and effective interface to AllAnime site.
"""
PROVIDER = "allanime"
api_endpoint = ALLANIME_API_ENDPOINT
HEADERS = {
"Referer": ALLANIME_REFERER,
@@ -55,7 +56,7 @@ class AllAnimeAPI(AnimeProvider):
logger.error("[ALLANIME-ERROR]: ", response.text)
return {}
@debug_provider("ALLANIME")
@debug_provider(PROVIDER.upper())
def search_for_anime(
self,
user_query: str,
@@ -106,7 +107,7 @@ class AllAnimeAPI(AnimeProvider):
}
return normalized_search_results
@debug_provider("ALLANIME")
@debug_provider(PROVIDER.upper())
def get_anime(self, allanime_show_id: str):
"""get an anime details given its id
@@ -121,6 +122,7 @@ class AllAnimeAPI(AnimeProvider):
id: str = anime["show"]["_id"]
title: str = anime["show"]["name"]
availableEpisodesDetail = anime["show"]["availableEpisodesDetail"]
self.store.set(allanime_show_id, "anime_info", {"title": title})
type = anime.get("__typename")
normalized_anime = {
"id": id,
@@ -130,9 +132,9 @@ class AllAnimeAPI(AnimeProvider):
}
return normalized_anime
@debug_provider("ALLANIME")
@debug_provider(PROVIDER.upper())
def _get_anime_episode(
self, allanime_show_id: str, episode_string: str, translation_type: str = "sub"
self, allanime_show_id: str, episode, translation_type: str = "sub"
) -> "AllAnimeEpisode | dict":
"""get the episode details and sources info
@@ -147,14 +149,14 @@ class AllAnimeAPI(AnimeProvider):
variables = {
"showId": allanime_show_id,
"translationType": translation_type,
"episodeString": episode_string,
"episodeString": episode,
}
episode = self._fetch_gql(ALLANIME_EPISODES_GQL, variables)
return episode["episode"]
@debug_provider("ALLANIME")
@debug_provider(PROVIDER.upper())
def get_episode_streams(
self, anime_id, anime_title, episode_number: str, translation_type="sub"
self, anime_id, episode_number: str, translation_type="sub"
):
"""get the streams of an episode
@@ -166,6 +168,10 @@ class AllAnimeAPI(AnimeProvider):
Yields:
[TODO:description]
"""
anime_title = (self.store.get(anime_id, "anime_info", "") or {"title": ""})[
"title"
]
allanime_episode = self._get_anime_episode(
anime_id, episode_number, translation_type
)
@@ -174,7 +180,7 @@ class AllAnimeAPI(AnimeProvider):
embeds = allanime_episode["sourceUrls"]
@debug_provider("ALLANIME")
@debug_provider(self.PROVIDER.upper())
def _get_server(embed):
# filter the working streams no need to get all since the others are mostly hsl
# TODO: should i just get all the servers and handle the hsl??

View File

@@ -21,7 +21,7 @@ from .constants import (
from .utils import process_animepahe_embed_page
if TYPE_CHECKING:
from .types import AnimePaheAnimePage, AnimePaheSearchPage, AnimeSearchResult
from .types import AnimePaheAnimePage, AnimePaheSearchPage, AnimePaheSearchResult
JUICY_STREAM_REGEX = re.compile(r"source='(.*)';")
logger = logging.getLogger(__name__)
@@ -32,8 +32,9 @@ class AnimePaheApi(AnimeProvider):
search_page: "AnimePaheSearchPage"
anime: "AnimePaheAnimePage"
HEADERS = REQUEST_HEADERS
PROVIDER = "animepahe"
@debug_provider("ANIMEPAHE")
@debug_provider(PROVIDER.upper())
def search_for_anime(self, user_query: str, *args):
url = f"{ANIMEPAHE_ENDPOINT}m=search&q={user_query}"
response = self.session.get(
@@ -43,6 +44,12 @@ class AnimePaheApi(AnimeProvider):
return
data: "AnimePaheSearchPage" = response.json()
self.search_page = data
for animepahe_search_result in data["data"]:
self.store.set(
str(animepahe_search_result["session"]),
"search_result",
animepahe_search_result,
)
return {
"pageInfo": {
@@ -66,96 +73,98 @@ class AnimePaheApi(AnimeProvider):
],
}
@debug_provider("ANIMEPAHE")
@debug_provider(PROVIDER.upper())
def get_anime(self, session_id: str, *args):
page = 1
anime_result: "AnimeSearchResult" = [
anime
for anime in self.search_page["data"]
if anime["session"] == session_id
][0]
data: "AnimePaheAnimePage" = {} # pyright:ignore
if d := self.store.get(str(session_id), "search_result"):
anime_result: "AnimePaheSearchResult" = d
data: "AnimePaheAnimePage" = {} # pyright:ignore
url = f"{ANIMEPAHE_ENDPOINT}m=release&id={session_id}&sort=episode_asc&page={page}"
url = f"{ANIMEPAHE_ENDPOINT}m=release&id={session_id}&sort=episode_asc&page={page}"
def _pages_loader(
url,
page,
):
response = self.session.get(
def _pages_loader(
url,
)
if response.ok:
if not data:
data.update(response.json())
else:
if ep_data := response.json().get("data"):
data["data"].extend(ep_data)
if response.json()["next_page_url"]:
# TODO: Refine this
time.sleep(
random.choice(
[
0.25,
0.1,
0.5,
0.75,
1,
]
page,
):
response = self.session.get(
url,
)
if response.ok:
if not data:
data.update(response.json())
else:
if ep_data := response.json().get("data"):
data["data"].extend(ep_data)
if response.json()["next_page_url"]:
# TODO: Refine this
time.sleep(
random.choice(
[
0.25,
0.1,
0.5,
0.75,
1,
]
)
)
page += 1
url = f"{ANIMEPAHE_ENDPOINT}m=release&id={session_id}&sort=episode_asc&page={page}"
_pages_loader(
url,
page,
)
)
page += 1
url = f"{ANIMEPAHE_ENDPOINT}m=release&id={session_id}&sort=episode_asc&page={page}"
_pages_loader(
url,
page,
)
_pages_loader(
url,
page,
)
_pages_loader(
url,
page,
)
if not data:
return {}
self.anime = data # pyright:ignore
episodes = list(map(str, [episode["episode"] for episode in data["data"]]))
title = ""
return {
"id": session_id,
"title": anime_result["title"],
"year": anime_result["year"],
"season": anime_result["season"],
"poster": anime_result["poster"],
"score": anime_result["score"],
"availableEpisodesDetail": {
"sub": episodes,
"dub": episodes,
"raw": episodes,
},
"episodesInfo": [
{
"title": f"{episode['title'] or title};{episode['episode']}",
"episode": episode["episode"],
"id": episode["session"],
"translation_type": episode["audio"],
"duration": episode["duration"],
"poster": episode["snapshot"],
}
for episode in data["data"]
],
}
if not data:
return {}
data["title"] = anime_result["title"] # pyright:ignore
self.store.set(str(session_id), "anime_info", data)
episodes = list(map(str, [episode["episode"] for episode in data["data"]]))
title = ""
return {
"id": session_id,
"title": anime_result["title"],
"year": anime_result["year"],
"season": anime_result["season"],
"poster": anime_result["poster"],
"score": anime_result["score"],
"availableEpisodesDetail": {
"sub": episodes,
"dub": episodes,
"raw": episodes,
},
"episodesInfo": [
{
"title": f"{episode['title'] or title};{episode['episode']}",
"episode": episode["episode"],
"id": episode["session"],
"translation_type": episode["audio"],
"duration": episode["duration"],
"poster": episode["snapshot"],
}
for episode in data["data"]
],
}
@debug_provider("ANIMEPAHE")
@debug_provider(PROVIDER.upper())
def get_episode_streams(
self, anime_id, anime_title, episode_number: str, translation_type, *args
self, anime_id, episode_number: str, translation_type, *args
):
anime_title = ""
episode = None
# extract episode details from memory
episode = [
episode
for episode in self.anime["data"]
if float(episode["episode"]) == float(episode_number)
]
if d := self.store.get(str(anime_id), "anime_info"):
anime_title = d["title"]
episode = [
episode
for episode in d["data"]
if float(episode["episode"]) == float(episode_number)
]
if not episode:
logger.error(f"[ANIMEPAHE-ERROR]: episode {episode_number} doesn't exist")
@@ -195,7 +204,7 @@ class AnimePaheApi(AnimeProvider):
continue
if not embed_url:
logger.warn(
logger.warning(
"[ANIMEPAHE-WARN]: embed url not found please report to the developers"
)
return []

View File

@@ -1,7 +1,7 @@
from typing import Literal, TypedDict
class AnimeSearchResult(TypedDict):
class AnimePaheSearchResult(TypedDict):
id: int
title: str
type: str
@@ -21,7 +21,7 @@ class AnimePaheSearchPage(TypedDict):
last_page: int
_from: int
to: int
data: list[AnimeSearchResult]
data: list[AnimePaheSearchResult]
class Episode(TypedDict):

View File

@@ -1,13 +1,34 @@
import os
import requests
from yt_dlp.utils.networking import random_user_agent
from ...constants import APP_CACHE_DIR
from .providers_store import ProviderStore
class AnimeProvider:
session: requests.Session
PROVIDER = ""
USER_AGENT = random_user_agent()
HEADERS = {}
def __init__(self) -> None:
self.session = requests.session()
def __init__(self, cache_requests, use_persistent_provider_store) -> None:
if cache_requests.lower() == "true":
from ..common.requests_cacher import CachedRequestsSession
self.session = CachedRequestsSession(
os.path.join(APP_CACHE_DIR, "cached_requests.db")
)
else:
self.session = requests.session()
self.session.headers.update({"User-Agent": self.USER_AGENT, **self.HEADERS})
if use_persistent_provider_store.lower() == "true":
self.store = ProviderStore(
"persistent",
self.PROVIDER,
os.path.join(APP_CACHE_DIR, "anime_providers_store.db"),
)
else:
self.store = ProviderStore("memory")

View File

@@ -41,7 +41,9 @@ class ParseAnchorAndImgTag(HTMLParser):
class HiAnimeApi(AnimeProvider):
# HEADERS = {"Referer": "https://hianime.to/home"}
@debug_provider("HIANIME")
PROVIDER = "hianime"
@debug_provider(PROVIDER.upper())
def search_for_anime(self, anime_title: str, *args):
query = quote_plus(anime_title)
url = f"https://hianime.to/search?keyword={query}"
@@ -77,157 +79,165 @@ class HiAnimeApi(AnimeProvider):
anime_id = anime_link_data["data-id"]
title = anime_link_data["title"]
results.append(
{
"availableEpisodes": list(range(1, episodes)),
"id": anime_id,
"title": title,
"poster": image_link,
}
)
self.search_results = results
return {"pageInfo": {}, "results": results}
@debug_provider("HIANIME")
def get_anime(self, hianime_id, *args):
anime_result = {}
for anime in self.search_results:
if anime["id"] == hianime_id:
anime_result = anime
break
anime_url = f"https://hianime.to/ajax/v2/episode/list/{hianime_id}"
response = self.session.get(anime_url, timeout=10)
if response.ok:
response_json = response.json()
hianime_anime_page = response_json["html"]
episodes_info_container_html = get_element_html_by_class(
"ss-list", hianime_anime_page
)
episodes_info_html_list = get_elements_html_by_class(
"ep-item", episodes_info_container_html
)
# keys: [ data-number: episode_number, data-id: episode_id, title: episode_title , href:episode_page_url]
episodes_info_dicts = [
extract_attributes(episode_dict)
for episode_dict in episodes_info_html_list
]
episodes = [episode["data-number"] for episode in episodes_info_dicts]
self.episodes_info = [
{
"id": episode["data-id"],
"title": (
(episode["title"] or "").replace(
f"Episode {episode['data-number']}", ""
)
or anime_result["title"]
)
+ f"; Episode {episode['data-number']}",
"episode": episode["data-number"],
}
for episode in episodes_info_dicts
]
return {
"id": hianime_id,
"availableEpisodesDetail": {
"dub": episodes,
"sub": episodes,
"raw": episodes,
},
"poster": anime_result["poster"],
"title": anime_result["title"],
"episodes_info": self.episodes_info,
result = {
"availableEpisodes": list(range(1, episodes)),
"id": anime_id,
"title": title,
"poster": image_link,
}
@debug_provider("HIANIME")
def get_episode_streams(
self, anime_id, anime_title, episode, translation_type, *args
):
episode_details = [
episode_details
for episode_details in self.episodes_info
if episode_details["episode"] == episode
]
if not episode_details:
return
episode_details = episode_details[0]
episode_url = f"https://hianime.to/ajax/v2/episode/servers?episodeId={episode_details['id']}"
response = self.session.get(episode_url)
if response.ok:
response_json = response.json()
episode_page_html = response_json["html"]
servers_containers_html = get_elements_html_by_class(
"ps__-list", episode_page_html
)
if not servers_containers_html:
return
# sub servers
try:
servers_html_sub = get_elements_html_by_class(
"server-item", servers_containers_html[0]
results.append(result)
self.store.set(result["id"], "search_result", result)
return {"pageInfo": {}, "results": results}
@debug_provider(PROVIDER.upper())
def get_anime(self, hianime_id, *args):
anime_result = {}
if d := self.store.get(str(hianime_id), "search_result"):
anime_result = d
anime_url = f"https://hianime.to/ajax/v2/episode/list/{hianime_id}"
response = self.session.get(anime_url, timeout=10)
if response.ok:
response_json = response.json()
hianime_anime_page = response_json["html"]
episodes_info_container_html = get_element_html_by_class(
"ss-list", hianime_anime_page
)
except Exception:
logger.warning("HiAnime: sub not found")
servers_html_sub = None
# dub servers
try:
servers_html_dub = get_elements_html_by_class(
"server-item", servers_containers_html[1]
episodes_info_html_list = get_elements_html_by_class(
"ep-item", episodes_info_container_html
)
except Exception:
logger.warning("HiAnime: dub not found")
servers_html_dub = None
if translation_type == "dub":
servers_html = servers_html_dub
else:
servers_html = servers_html_sub
if not servers_html:
return
@debug_provider("HIANIME")
def _get_server(server_name, server_html):
# keys: [ data-type: translation_type, data-id: embed_id, data-server-id: server_id ]
servers_info = extract_attributes(server_html)
embed_url = f"https://hianime.to/ajax/v2/episode/sources?id={servers_info['data-id']}"
embed_response = self.session.get(embed_url)
if embed_response.ok:
embed_json = embed_response.json()
raw_link_to_streams = embed_json["link"]
match = LINK_TO_STREAMS_REGEX.match(raw_link_to_streams)
if not match:
return
provider_domain = match.group(1)
embed_type = match.group(2)
episode_number = match.group(3)
source_id = match.group(4)
link_to_streams = f"https://{provider_domain}/embed-{embed_type}/ajax/e-{episode_number}/getSources?id={source_id}"
link_to_streams_response = self.session.get(link_to_streams)
if link_to_streams_response.ok:
juicy_streams_json: "HiAnimeStream" = (
link_to_streams_response.json()
# keys: [ data-number: episode_number, data-id: episode_id, title: episode_title , href:episode_page_url]
episodes_info_dicts = [
extract_attributes(episode_dict)
for episode_dict in episodes_info_html_list
]
episodes = [episode["data-number"] for episode in episodes_info_dicts]
episodes_info = [
{
"id": episode["data-id"],
"title": (
(episode["title"] or "").replace(
f"Episode {episode['data-number']}", ""
)
or anime_result["title"]
)
return {
"headers": {},
"subtitles": [
{
"url": track["file"],
"language": track["label"],
}
for track in juicy_streams_json["tracks"]
if track["kind"] == "captions"
],
"server": server_name,
"episode_title": episode_details["title"],
"links": give_random_quality(
[
{"link": link["file"], "type": link["type"]}
for link in juicy_streams_json["sources"]
]
),
}
+ f"; Episode {episode['data-number']}",
"episode": episode["data-number"],
}
for episode in episodes_info_dicts
]
self.store.set(
str(hianime_id),
"anime_info",
episodes_info,
)
return {
"id": hianime_id,
"availableEpisodesDetail": {
"dub": episodes,
"sub": episodes,
"raw": episodes,
},
"poster": anime_result["poster"],
"title": anime_result["title"],
"episodes_info": episodes_info,
}
for server_name, server_html in zip(cycle(SERVERS_AVAILABLE), servers_html):
if server := _get_server(server_name, server_html):
yield server
@debug_provider(PROVIDER.upper())
def get_episode_streams(self, anime_id, episode, translation_type, *args):
if d := self.store.get(str(anime_id), "anime_info"):
episodes_info = d
episode_details = [
episode_details
for episode_details in episodes_info
if episode_details["episode"] == episode
]
if not episode_details:
return
episode_details = episode_details[0]
episode_url = f"https://hianime.to/ajax/v2/episode/servers?episodeId={episode_details['id']}"
response = self.session.get(episode_url)
if response.ok:
response_json = response.json()
episode_page_html = response_json["html"]
servers_containers_html = get_elements_html_by_class(
"ps__-list", episode_page_html
)
if not servers_containers_html:
return
# sub servers
try:
servers_html_sub = get_elements_html_by_class(
"server-item", servers_containers_html[0]
)
except Exception:
logger.warning("HiAnime: sub not found")
servers_html_sub = None
# dub servers
try:
servers_html_dub = get_elements_html_by_class(
"server-item", servers_containers_html[1]
)
except Exception:
logger.warning("HiAnime: dub not found")
servers_html_dub = None
if translation_type == "dub":
servers_html = servers_html_dub
else:
servers_html = servers_html_sub
if not servers_html:
return
@debug_provider(self.PROVIDER.upper())
def _get_server(server_name, server_html):
# keys: [ data-type: translation_type, data-id: embed_id, data-server-id: server_id ]
servers_info = extract_attributes(server_html)
embed_url = f"https://hianime.to/ajax/v2/episode/sources?id={servers_info['data-id']}"
embed_response = self.session.get(embed_url)
if embed_response.ok:
embed_json = embed_response.json()
raw_link_to_streams = embed_json["link"]
match = LINK_TO_STREAMS_REGEX.match(raw_link_to_streams)
if not match:
return
provider_domain = match.group(1)
embed_type = match.group(2)
episode_number = match.group(3)
source_id = match.group(4)
link_to_streams = f"https://{provider_domain}/embed-{embed_type}/ajax/e-{episode_number}/getSources?id={source_id}"
link_to_streams_response = self.session.get(link_to_streams)
if link_to_streams_response.ok:
juicy_streams_json: "HiAnimeStream" = (
link_to_streams_response.json()
)
# TODO: Hianime decided to fucking encrypt shit
# so got to fix it later
return {
"headers": {},
"subtitles": [
{
"url": track["file"],
"language": track["label"],
}
for track in juicy_streams_json["tracks"]
if track["kind"] == "captions"
],
"server": server_name,
"episode_title": episode_details["title"],
"links": give_random_quality(
[
{"link": link["file"]}
for link in juicy_streams_json["tracks"]
]
),
}
for server_name, server_html in zip(
cycle(SERVERS_AVAILABLE), servers_html
):
if server := _get_server(server_name, server_html):
yield server

View File

@@ -29,8 +29,9 @@ EXTRACT_USEFUL_INFO_PATTERN_2 = re.compile(
class NyaaApi(AnimeProvider):
search_results: SearchResults
PROVIDER = "nyaa"
@debug_provider("NYAA")
@debug_provider(PROVIDER.upper())
def search_for_anime(self, user_query: str, *args, **_):
self.search_results = search_for_anime_with_anilist(
user_query, True
@@ -38,7 +39,7 @@ class NyaaApi(AnimeProvider):
self.user_query = user_query
return self.search_results
@debug_provider("NYAA")
@debug_provider(PROVIDER.upper())
def get_anime(self, anilist_id: str, *_):
for anime in self.search_results["results"]:
if anime["id"] == anilist_id:
@@ -54,11 +55,10 @@ class NyaaApi(AnimeProvider):
},
}
@debug_provider("NYAA")
@debug_provider(PROVIDER.upper())
def get_episode_streams(
self,
anime_id: str,
anime_title: str,
episode_number: str,
translation_type: str,
trusted_only=bool(int(os.environ.get("FA_NYAA_TRUSTED_ONLY", "0"))),
@@ -66,6 +66,7 @@ class NyaaApi(AnimeProvider):
sort_by="seeders",
*args,
):
anime_title = self.titles[0]
logger.debug(f"Searching nyaa for query: '{anime_title} {episode_number}'")
servers = {}

View File

@@ -0,0 +1,114 @@
import json
import logging
import time
logger = logging.getLogger(__name__)
class ProviderStoreDB:
def __init__(
self,
provider_name,
cache_db_path: str,
max_lifetime: int = 604800,
max_size: int = (1024**2) * 10,
table_name: str = "fastanime_providers_store",
clean_db=False,
):
from ..common.sqlitedb_helper import SqliteDB
self.cache_db_path = cache_db_path
self.clean_db = clean_db
self.provider_name = provider_name
self.max_lifetime = max_lifetime
self.max_size = max_size
self.table_name = table_name
self.sqlite_db_connection = SqliteDB(self.cache_db_path)
# Prepare the cache table if it doesn't exist
self._create_store_table()
def _create_store_table(self):
"""Create cache table if it doesn't exist."""
with self.sqlite_db_connection as conn:
conn.execute(
f"""
CREATE TABLE IF NOT EXISTS {self.table_name} (
id TEXT,
data_type TEXT,
provider_name TEXT,
data TEXT,
cache_expiry INTEGER
)"""
)
def get(self, id: str, data_type: str, default=None):
with self.sqlite_db_connection as conn:
cursor = conn.cursor()
cursor.execute(
f"""
SELECT
data
FROM {self.table_name}
WHERE
id = ?
AND data_type = ?
AND provider_name = ?
AND cache_expiry > ?
""",
(id, data_type, self.provider_name, int(time.time())),
)
cached_data = cursor.fetchone()
if cached_data:
logger.debug("Found existing request in cache")
(json_data,) = cached_data
return json.loads(json_data)
return default
def set(self, id: str, data_type: str, data):
with self.sqlite_db_connection as connection:
cursor = connection.cursor()
cursor.execute(
f"""
INSERT INTO {self.table_name}
VALUES ( ?, ?,?, ?, ?)
""",
(
id,
data_type,
self.provider_name,
json.dumps(data),
int(time.time()) + self.max_lifetime,
),
)
class ProviderStoreMem:
def __init__(self) -> None:
from collections import defaultdict
self._store = defaultdict(dict)
def get(self, id: str, data_type: str, default=None):
return self._store[id][data_type]
def set(self, id: str, data_type: str, data):
self._store[id][data_type] = data
def ProviderStore(store_type, *args, **kwargs):
if store_type == "persistent":
return ProviderStoreDB(*args, **kwargs)
else:
return ProviderStoreMem()
if __name__ == "__main__":
store = ProviderStore("persistent", "test_provider", "provider_store")
store.set("123", "test", {"hello": "world"})
print(store.get("123", "test"))
print("-------------------------------")
store = ProviderStore("memory")
store.set("1", "test", {"hello": "world"})
print(store.get("1", "test"))

View File

@@ -176,11 +176,17 @@ query ($query: String) {
if not anime_result["status"] == "RELEASING"
and anime_result["episodes"]
else (
anime_result["nextAiringEpisode"]["episode"] - 1
if anime_result["nextAiringEpisode"]
else 0
(
anime_result["nextAiringEpisode"]["episode"]
- 1
if anime_result["nextAiringEpisode"]
else 0
)
if not anime_result["episodes"]
else anime_result["episodes"]
)
),
)
+ 1,
),
)
),

View File

@@ -0,0 +1,214 @@
import json
import logging
import re
import time
from datetime import datetime
from urllib.parse import urlencode
import requests
from .sqlitedb_helper import SqliteDB
logger = logging.getLogger(__name__)
caching_mimetypes = {
"application": {
"json",
"xml",
"x-www-form-urlencoded",
"x-javascript",
"javascript",
},
"text": {"html", "css", "javascript", "plain", "xml", "xsl", "x-javascript"},
}
class CachedRequestsSession(requests.Session):
__request_functions__ = (
"get",
"options",
"head",
"post",
"put",
"patch",
"delete",
)
def __new__(cls, *args, **kwargs):
def caching_params(name: str):
def wrapper(self, *args, **kwargs):
return cls.request(self, name, *args, **kwargs)
return wrapper
for func in cls.__request_functions__:
setattr(cls, func, caching_params(func))
return super().__new__(cls)
def __init__(
self,
cache_db_path: str,
max_lifetime: int = 604800,
max_size: int = (1024**2) * 10,
table_name: str = "fastanime_requests_cache",
clean_db=False,
*args,
**kwargs,
):
super().__init__(*args, **kwargs)
self.cache_db_path = cache_db_path
self.max_lifetime = max_lifetime
self.max_size = max_size
self.table_name = table_name
self.sqlite_db_connection = SqliteDB(self.cache_db_path)
# Prepare the cache table if it doesn't exist
self._create_cache_table()
def _create_cache_table(self):
"""Create cache table if it doesn't exist."""
with self.sqlite_db_connection as conn:
conn.execute(
f"""
CREATE TABLE IF NOT EXISTS {self.table_name} (
url TEXT,
status_code INTEGER,
request_headers TEXT,
response_headers TEXT,
data BLOB,
redirection_policy INT,
cache_expiry INTEGER
)"""
)
def request(
self,
method,
url,
params=None,
force_caching=False,
fresh=0,
*args,
**kwargs,
):
# TODO: improve the caching functionality and add a layer to auto delete
# expired requests
if fresh:
logger.debug("Executing fresh request")
return super().request(method, url, params=params, *args, **kwargs)
if params:
url += "?" + urlencode(params)
redirection_policy = int(kwargs.get("force_redirects", False))
with self.sqlite_db_connection as conn:
cursor = conn.cursor()
time_before_access_db = datetime.now()
logger.debug("Checking for existing request in cache")
cursor.execute(
f"""
SELECT
status_code,
request_headers,
response_headers,
data,
redirection_policy
FROM {self.table_name}
WHERE
url = ?
AND redirection_policy = ?
AND cache_expiry > ?
""",
(url, redirection_policy, int(time.time())),
)
cached_request = cursor.fetchone()
time_after_access_db = datetime.now()
if cached_request:
logger.debug("Found existing request in cache")
(
status_code,
request_headers,
response_headers,
data,
redirection_policy,
) = cached_request
response = requests.Response()
response.headers.update(json.loads(response_headers))
response.status_code = status_code
response._content = data
if "timeout" in kwargs:
kwargs.pop("timeout")
if "headers" in kwargs:
kwargs.pop("headers")
_request = requests.Request(
method, url, headers=json.loads(request_headers), *args, **kwargs
)
response.request = _request.prepare()
response.elapsed = time_after_access_db - time_before_access_db
return response
# Perform the request and cache it
response = super().request(method, url, *args, **kwargs)
if response.ok and (
force_caching
or self.is_content_type_cachable(
response.headers.get("content-type"), caching_mimetypes
)
and len(response.content) < self.max_size
):
logger.debug("Caching the current request")
cursor.execute(
f"""
INSERT INTO {self.table_name}
VALUES (?, ?, ?, ?, ?, ?, ?)
""",
(
url,
response.status_code,
json.dumps(dict(response.request.headers)),
json.dumps(dict(response.headers)),
response.content,
redirection_policy,
int(time.time()) + self.max_lifetime,
),
)
return response
@staticmethod
def is_content_type_cachable(content_type, caching_mimetypes):
"""Checks whether the given encoding is supported by the cacher"""
if content_type is None:
return True
mime, contents = content_type.split("/")
contents = re.sub(r";.*$", "", contents)
return mime in caching_mimetypes and any(
content in caching_mimetypes[mime] for content in contents.split("+")
)
if __name__ == "__main__":
with CachedRequestsSession("cache.db") as session:
response = session.get(
"https://google.com",
)
response_b = session.get(
"https://google.com",
)
print("A: ", response.elapsed)
print("B: ", response_b.elapsed)
print(response_b.text[0:30])

View File

@@ -0,0 +1,34 @@
import logging
import sqlite3
import time
logger = logging.getLogger(__name__)
class SqliteDB:
def __init__(self, db_path: str) -> None:
self.db_path = db_path
self.connection = sqlite3.connect(self.db_path)
logger.debug("Enabling WAL mode for concurrent access")
self.connection.execute("PRAGMA journal_mode=WAL;")
self.connection.close()
self.connection = None
def __enter__(self):
logger.debug("Starting new connection...")
start_time = time.time()
self.connection = sqlite3.connect(self.db_path)
logger.debug(
"Successfully got a new connection in {} seconds".format(
time.time() - start_time
)
)
return self.connection
def __exit__(self, exc_type, exc_val, exc_tb):
if self.connection:
logger.debug("Closing connection to cache db")
self.connection.commit()
self.connection.close()
self.connection = None
logger.debug("Successfully closed connection to cache db")

View File

@@ -49,7 +49,7 @@ class FZF:
"--info=hidden",
"--layout=reverse",
"--height=100%",
"--bind=right:accept",
"--bind=right:accept,ctrl-/:toggle-preview,ctrl-space:toggle-wrap+toggle-preview-wrap",
"--no-margin",
"+m",
"-i",

11
make_release Executable file
View File

@@ -0,0 +1,11 @@
#! /usr/bin/env sh
CLI_DIR="$(dirname "$(realpath "$0")")"
VERSION=$1
[ -z "$VERSION" ] && echo no version provided && exit 1
[ "$VERSION" = "current" ] && fastanime --version && exit 0
sed -i "s/^version.*/version = \"$VERSION\"/" "$CLI_DIR/pyproject.toml" &&
sed -i "s/__version__.*/__version__ = \"v$VERSION\"/" "$CLI_DIR/fastanime/__init__.py" &&
git stage "$CLI_DIR/pyproject.toml" "$CLI_DIR/fastanime/__init__.py" &&
git commit -m "chore: bump version (v$VERSION)" &&
git push &&
gh release create "v$VERSION"

1024
poetry.lock generated

File diff suppressed because it is too large Load Diff

View File

@@ -1,6 +1,6 @@
[tool.poetry]
name = "fastanime"
version = "2.5.6.dev1"
version = "2.6.5"
description = "A browser anime site experience from the terminal"
authors = ["Benextempest <benextempest@gmail.com>"]
license = "UNLICENSE"
@@ -17,12 +17,13 @@ inquirerpy = { version = "^0.3.4", optional = false }
mpv = { version = "^1.0.7", optional = true }
plyer = { version = "^2.1.0", optional = true }
lbry-libtorrent = "^1.2.4"
fastapi = {extras = ["standard"], version = "^0.115.0", optional = true}
[tool.poetry.extras]
full = ["plyer", "mpv"]
full = ["plyer", "mpv", "fastapi"]
# cli = ["rich", "click", "inquirerpy"]
mpv = ["mpv"]
notifications = ["plyer"]
api = ["fastapi"]
[tool.poetry.group.dev.dependencies]
black = "^24.4.2"

View File

@@ -6,7 +6,7 @@ from fastanime.cli import run_cli
@pytest.fixture
def runner():
return CliRunner()
return CliRunner(env={"FASTANIME_CACHE_REQUESTS": "false"})
def test_main_help(runner: CliRunner):