unidown.plugin¶

unidown.plugin.a_plugin¶

class unidown.plugin.a_plugin.APlugin(settings, options=None)[source]¶

Abstract class of a plugin. Provides all needed variables and methods.

Parameters

options (Optional[Dict[str, Any]]) – parameters which can included optional parameters

Raises

PluginException – can not create default plugin paths

Variables

_info – information about the plugin
_savestate_cls – savestate class to use
_disable_tqdm – if the tqdm progressbar should be disabled | do not edit
_log – use this for logging | do not edit
_simul_downloads – number of simultaneous downloads
_temp_dir – path where the plugin can place all temporary data | do not edit
_download_dir – general download path of the plugin | do not edit
_savestate_file – file which contains the latest savestate of the plugin | do not edit
_last_update – latest update time of the referencing data | do not edit
_unit – the thing which should be downloaded, may be displayed in the progress bar
_download_data – referencing data | do not edit
_downloader – downloader which will download the data | do not edit
_savestate – savestate of the plugin
_options – options which the plugin uses internal, should be used for the given options at init

abstract _create_download_data()[source]¶

Get the download links in a specific format. Has to be implemented inside Plugins.

Raises: NotImplementedError – abstract method
Return type: LinkItemDict

abstract _create_last_update_time()[source]¶

Get the newest update time from the referencing data. Has to be implemented inside Plugins.

Raises: NotImplementedError – abstract method
Return type: datetime

_load_default_options()[source]¶: Loads default options if they were not passed at creation.

_savestate_cls¶: alias of unidown.plugin.savestate.SaveState

check_download(link_item_dict, folder, log=False)[source]¶

Check if the download of the given dict was successful. No proving if the content of the file is correct too.

Parameters

link_item_dict (LinkItemDict) – dict which to check
folder (Path) – folder where the downloads are saved
log (bool) – if the lost items should be logged

Return type

Tuple[LinkItemDict, LinkItemDict]

Returns

succeeded and failed

clean_up()[source]¶: Default clean up for a module. Deletes _temp_dir.

download(link_items, folder, desc, unit)[source]¶

Warning

The parameters may change in future versions. (e.g. change order and accept another host)

Download the given LinkItem dict from the plugins host, to the given path. Proceeded with multiple connections _simul_downloads. After check_download() is recommend.

This function don’t use an internal link_item_dict, delay or folder directly set in options or instance vars, because it can be used aside of the normal download routine inside the plugin itself for own things. As of this it still needs access to the logger, so a staticmethod is not possible.

Parameters

link_items (LinkItemDict) – data which gets downloaded
folder (Path) – target download folder
desc (str) – description of the progressbar
unit (str) – unit of the download, shown in the progressbar

download_as_file(url, target_file, delay=0)[source]¶

Download the given url to the given target folder.

Parameters

url (str) – link
target_file (Path) – target file
delay (float) – after download wait this much seconds

Return type

str

Returns

url

Raises

HTTPError – if the connection has an error

property download_data¶

Plain getter.

Return type: LinkItemDict

property download_dir¶

Plain getter.

Return type: Path

static get_plugins()[source]¶

Get all available plugins for unidown.

Return type: Dict[str, EntryPoint]
Returns: plugin name list

property host¶

Plain getter.

Return type: str

property info¶

Plain getter.

Return type: PluginInfo

property last_update¶

Plain getter.

Return type: datetime

load_savestate()[source]¶

Load the save of the plugin.

Raises

PluginException – broken savestate json
PluginException – different savestate versions
PluginException – different plugin versions
PluginException – different plugin names
PluginException – could not parse the json

property log¶

Plain getter.

Return type: Logger

property name¶

Plain getter.

Return type: str

property options¶

Plain getter.

Return type: Dict[str, Any]

save_savestate()[source]¶: Save meta data about the downloaded things and the plugin to file.

property savestate¶: Plain getter.

property simul_downloads¶

Plain getter.

Return type: int

property temp_dir¶

Plain getter.

Return type: Path

property unit¶

Plain getter.

Return type: str

update_download_data()[source]¶: Update the download links. Calls _create_download_data().

update_last_update()[source]¶: Call this to update the latest update time. Calls _create_last_update_time().

update_savestate(new_items)[source]¶

Update savestate.

Parameters: new_items (LinkItemDict) – new items

property version¶

Plain getter.

Return type: Version

unidown.plugin.exceptions¶

Default exceptions of plugins.

exception unidown.plugin.exceptions.PluginException(msg='')[source]¶

Base class for exceptions in a plugin. If catching this, it implicit that the plugin is unable to work further.

Parameters: msg (str) – message
Variables: msg – exception message

unidown.plugin.link_item¶

class unidown.plugin.link_item.LinkItem(name, time)[source]¶

Item which represents the data, who need to be downloaded. Has a name and an update time.

Parameters

name (str) – name
time (datetime) – update time

Raises

ValueError – name cannot be empty or None
ValueError – time cannot be empty or None

Variables

time_format – time format to use
_name – name of the item
_time – time of the item

classmethod from_json(data)[source]¶

Constructor from json dict.

Parameters: data (dict) – json data as dict
Return type: LinkItem
Returns: the LinkItem
Raises: ValueError – missing parameter

property name¶

Plain getter.

Return type: str

property time¶

Plain getter.

Return type: datetime

to_json()[source]¶

Create json data.

Return type: dict
Returns: json dictionary

unidown.plugin.link_item_dict¶

class unidown.plugin.link_item_dict.LinkItemDict[source]¶

LinkItem dictionary, acts as a wrapper for special methods and functions.

actualize(new_data, log=None)[source]¶

Actualize dictionary like an ~dict.update does. If a logger is passed it will log updated items, not new one.

Parameters

new_data (LinkItemDict) – the data used for updating
log (Optional[Logger]) – logger

clean_up_names()[source]¶: Rename duplicated names with an additional _r.

static get_new_items(old_data, new_data, disable_tqdm=False)[source]¶

Get the new items which are not existing or are newer as in the old data set.

Parameters

old_data (LinkItemDict) – old data
new_data (LinkItemDict) – new data
disable_tqdm (bool) – disables tqdm progressbar

Return type

LinkItemDict

Returns

new and updated link items

unidown.plugin.plugin_info¶

class unidown.plugin.plugin_info.PluginInfo(name, version, host)[source]¶

Information about the module. Those information will be saved into the save files as well.

Parameters

name (str) – the name of the plugin
version (str) – version, PEP440 conform
host (str) – host url of the main data

Raises

ValueError – name is empty
ValueError – host is empty
InvalidVersion – version is not PEP440 conform

Variables

name – name of the plugin
host – host url of the main data
version – plugin version

classmethod from_json(data)[source]¶

Constructor from json dict.

Parameters

data (dict) – json data as dict

Return type

PluginInfo

Returns

the plugin info

Raises

ValueError – name is missing
ValueError – version is missing
ValueError – host is missing

to_json()[source]¶

Create json data.

Return type: dict
Returns: json dictionary

unidown.plugin.savestate¶

class unidown.plugin.savestate.SaveState(plugin_info, last_update, link_items, version=<Version('1')>)[source]¶

Savestate of a plugin.

Parameters

version (Version) – savestate version
plugin_info (PluginInfo) – plugin info
last_update (datetime) – last udpate time of the referenced data
link_items (LinkItemDict) – data
version – savestate version

Variables

time_format – time format to use
version – savestate version
plugin_info – plugin info
last_update – newest udpate time
link_items – data

classmethod from_json(data)[source]¶

Parameters

data (dict) – json data as dict

Return type

SaveState

Returns

the SaveState

Raises

ValueError – version of SaveState does not exist or is empty
InvalidVersion – version is not PEP440 conform

to_json()[source]¶

Create json data.

Return type: dict
Returns: json dictionary

upgrade()[source]¶

Upgrading current savestate to the latest savestate version.

Return type: SaveState
Returns: upgraded savestate