unidown.plugin¶

exception PluginError(msg='')[source]¶

Bases: Exception

Base class for exceptions in a plugin. If catching this, it is implicit that the plugin is unable to work further.

Parameters: msg (str) – message

msg: str¶: Exception message.

class APlugin(settings, options=None)[source]¶

Bases: ABC

Abstract class of a plugin. Provides all needed variables and methods.

Parameters

options (Optional[dict[str, Any]]) – Parameters which can include optional parameters.
settings (Settings) –

Raises

PluginError – Can not create default plugin paths.

_SAVESTATE_CLS¶: alias of SaveState

abstract _create_download_data()[source]¶

Get the download links in a specific format.

Note

Has to be implemented inside Plugins.

Raises: NotImplementedError – Abstract method.
Return type: LinkItemDict

abstract _create_last_update_time()[source]¶

Get the newest update time from the referencing data.

Note

Has to be implemented inside Plugin.

Raises: NotImplementedError – Abstract method.
Return type: datetime

_load_default_options()[source]¶

Load default options if they were not passed at creation.

Return type: None

check_download(link_item_dict, folder, log=False)[source]¶

Check if the download of the given dict was successful. No proving if the content of the file is correct too.

Parameters

link_item_dict (LinkItemDict) – Items to check.
folder (Path) – Folder where the downloads are saved.
log (bool) – Log lost items.

Return type

tuple[LinkItemDict, LinkItemDict]

Returns

Succeed.

clean_up()[source]¶

Clean up for a module. Is deleting _temp_dir.

Return type: None

download(link_items, folder, desc, unit)[source]¶

Download the given LinkItem dict from the plugins host, to the given path. Proceeded with multiple connections. _simul_downloads. After check_download() is recommended.

This function don’t use an internal link_item_dict, delay or folder directly set in options or instance vars, because it can be used aside of the normal download routine inside the plugin itself for own things. As of this it still needs access to the logger, so a staticmethod is not possible.

Warning

The parameters may change in future versions. (e.g. change order and accept another host)

Parameters

link_items (LinkItemDict) – Data which gets downloaded.
folder (Path) – Target download folder.
desc (str) – Description of the progressbar.
unit (str) – Unit of the download, shown in the progressbar.

Return type

None

download_as_file(url, target_file, delay=0)[source]¶

Download the given url to the given target folder.

Parameters

url (str) – Link.
target_file (Path) – Target file.
delay (float) – Delay after each download. Delay is in seconds.

Return type

str

Returns

Url.

Raises

HTTPError – Connection had an error.

load_savestate()[source]¶

Load the save of the plugin.

Raises

PluginError – Broken savestate json.
PluginError – Different savestate versions.
PluginError – Different plugin versions.
PluginError – Different plugin names.
PluginError – Could not parse the json.

Return type

None

save_savestate()[source]¶

Save meta data about the downloaded things and the plugin to a file.

Return type: None

update_download_data()[source]¶

Update the download links. Calls _create_download_data().

Return type: None

update_last_update()[source]¶

Call this to update the latest update time. Calls _create_last_update_time().

Return type: None

update_savestate(new_items)[source]¶

Update savestate.

Parameters: new_items (LinkItemDict) – New items.
Return type: None

_INFO: PluginInfo = <unidown.plugin.plugin_info.PluginInfo object>¶: Meta information about the plugin.

_abc_impl = <_abc._abc_data object>¶

_disable_tqdm: bool¶: If the tqdm progressbar should be disabled.

_download_data: LinkItemDict¶: Referencing data.

_download_dir: Path¶: General download path of the plugin.

_downloader: HTTPSConnectionPool¶: Downloader which will download the data.

_last_update: datetime¶: Latest update time of the referencing data.

_log: Logger¶: Use this for logging.

_options: dict[str, Any]¶: Options which the plugin uses internally, should be used for the given options at initialization.

_savestate: SaveState¶: Savestate of the plugin.

_savestate_file: Path¶: File which contains the latest savestate of the plugin.

_simul_downloads: int¶: Number of simultaneous downloads.

_temp_dir: Path¶: Path where the plugin can place all temporary data.

_unit: str¶: The unit which will be displayed in the progress bar.

property download_data: LinkItemDict¶

Plain getter.

Return type: LinkItemDict

property download_dir: Path¶

Plain getter.

Return type: Path

property host: str¶

Plain getter.

Return type: str

property info: PluginInfo¶

Plain getter.

Return type: PluginInfo

property last_update: datetime¶

Plain getter.

Return type: datetime

property log: Logger¶

Plain getter.

Return type: Logger

property name: str¶

Plain getter.

Return type: str

property options: dict[str, Any]¶

Plain getter.

Return type: dict[str, Any]

property savestate: SaveState¶

Plain getter.

Return type: SaveState

property simul_downloads: int¶

Plain getter.

Return type: int

property temp_dir: Path¶

Plain getter.

Return type: Path

property unit: str¶

Plain getter.

Return type: str

property version: Version¶

Plain getter.

Return type: Version

class LinkItem(name, time)[source]¶

Bases: object

Item which represents the data, who need to be downloaded. Has a name and an update time.

Parameters

name (str) – name
time (datetime) – update time

Raises

ValueError – name cannot be empty or None
ValueError – time cannot be empty or None

classmethod from_json(data)[source]¶

Construct from json dict.

Parameters: data (dict) – Json data as dict.
Return type: LinkItem
Returns: LinkItem.
Raises: ValueError – Missing parameter.

to_json()[source]¶

Create json data.

Return type: dict
Returns: Json dictionary.

TIME_FORMAT: str = '%Y%m%dT%H%M%S.%fZ'¶: Time format to use.

_name: str¶: Name of the item.

_time: datetime¶: Time of the item.

property name: str¶

Plain getter.

Return type: str

property time: datetime¶

Plain getter.

Return type: datetime

class LinkItemDict[source]¶

Bases: dict

LinkItem dictionary, acts as a wrapper for special methods and functions.

actualize(new_data, log=None)[source]¶

Actualize dictionary like ~dict.update does. If a logger is passed it will log updated items, not new one.

Parameters

new_data (LinkItemDict) – Data used for updating.
log (Optional[Logger]) – Logger.

Return type

None

clean_up_names()[source]¶

Rename duplicated names with an additional _r.

Return type: None

static get_new_items(old_data, new_data, disable_tqdm=False)[source]¶

Get the new items which are not existing or are newer as in the old data set.

Parameters

old_data (LinkItemDict) – Old data.
new_data (LinkItemDict) – New data.
disable_tqdm (bool) – Disable tqdm progressbar.

Return type

LinkItemDict

Returns

New and updated link items.

class PluginInfo(name, version, host)[source]¶

Bases: object

Information about the module. This information will be saved into the save files as well.

Parameters

name (str) – Name of the plugin.
version (str) – Version, PEP440 conform.
host (str) – Host url of the main data.

Raises

ValueError – Name is empty.
ValueError – Host is empty.
InvalidVersion – Version is not PEP440 conform.

classmethod from_json(data)[source]¶

Construct from json dict.

Parameters

data (dict) – Json data as dict.

Return type

PluginInfo

Returns

Plugin info.

Raises

ValueError – Name is missing.
ValueError – Version is missing.
ValueError – Host is missing.

to_json()[source]¶

Create json data.

Return type: dict
Returns: Json dictionary.

host: str¶: Host url of the main data.

name: str¶: Name of the plugin.

version: Version¶: Plugin version.

class SaveState(plugin_info, last_update, link_items, version=<Version('1')>)[source]¶

Bases: object

Savestate of a plugin.

Parameters

version (Version) – Savestate version.
plugin_info (PluginInfo) – Plugin info.
last_update (datetime) – Last udpate time of the referenced data.
link_items (LinkItemDict) – Data.
version – Savestate version.

classmethod from_json(data)[source]¶

Parameters

data (dict) – Json data as dict.

Return type

SaveState

Returns

SaveState.

Raises

ValueError – Version of SaveState does not exist or is empty.
InvalidVersion – Version is not PEP440 conform.

to_json()[source]¶

Create json data.

Return type: dict
Returns: Json dictionary.

upgrade()[source]¶

Upgrading current savestate to the latest savestate version.

Return type: SaveState
Returns: Upgraded savestate.

TIME_FORMAT: str = '%Y%m%dT%H%M%S.%fZ'¶: Time format to use.

_DEFAULT_VERSION: Version = <Version('1')>¶: Default savestate version.

last_update: datetime¶: Newest update time.

link_items: LinkItemDict¶: Data.

plugin_info: PluginInfo¶: Plugin info.

version: Version¶: Savestate version.

get_plugins()[source]¶

Get all available plugins for unidown.

Return type: dict[str, EntryPoint]
Returns: Plugin name list.

Modules

`unidown.plugin.a_plugin`
`unidown.plugin.exceptions`	Default exceptions of plugins.
`unidown.plugin.link_item`
`unidown.plugin.link_item_dict`
`unidown.plugin.plugin_info`
`unidown.plugin.savestate`