8.1. Data Processors¶

8.1.1. DataProcessor¶

class easydata.processors.data.DataProcessor(source: str = 'main', new_source: Optional[str] = None, process_source_data=None, debug: bool = False, debug_raw: bool = False)[source]¶: Bases: easydata.processors.data.DataBaseProcessor

DataProcessor directly extends DataBaseProcessor and it’s default parameters. On it’s own, it doesn’t add any new functionalities besides those that are inherited from a DataBaseProcessor.

Note

All other data processors are based on DataBaseProcessor and if you are creating your own, then you should also extend from DataBaseProcessor. Also DataBaseProcessor cannot be used in a model, since it’s an abstract class.

Parameters¶

source¶

Default value of source parameter is data. This means that by default DataProcessor will always look into DataBag for a data key and it’s content. If we need to modify content from a different source in a DataBag, then we just need to change source value.

new_source¶

By default content that is getting processed from the source value, will be overwritten in a DataBag under key that is specified by source parameter.

We can still preserve original content in a DataBag, by specifying new source name under a new_source parameter.

Note

If we store modified content under a new key in a DataBag, parsers by default will look into a data key source and we need to specify that under a parsers parameter source, which name is same as key from a new_source in a data processor property.

process_source_data¶

8.1.2. DataToPqProcessor¶

class easydata.processors.data.DataToPqProcessor(source: str = 'main', new_source: Optional[str] = None, process_source_data=None, debug: bool = False, debug_raw: bool = False)[source]¶: Bases: easydata.processors.data.DataBaseProcessor

8.1.3. DataJsonToDictProcessor¶

class easydata.processors.data.DataJsonToDictProcessor(source: str = 'main', new_source: Optional[str] = None, process_source_data=None, debug: bool = False, debug_raw: bool = False)[source]¶: Bases: easydata.processors.data.DataBaseProcessor

8.1.4. DataYamlToDictProcessor¶

class easydata.processors.data.DataYamlToDictProcessor(*args, safe_load: bool = True, **kwargs)[source]¶: Bases: easydata.processors.data.DataBaseProcessor

8.1.5. DataJsonFromQueryToDictProcessor¶

class easydata.processors.data.DataJsonFromQueryToDictProcessor(query: Union[easydata.queries.base.QuerySearchBase, easydata.parsers.base.BaseData], **kwargs)[source]¶: Bases: easydata.processors.data.DataFromQueryProcessor

8.1.6. DataXmlToDictProcessor¶

class easydata.processors.data.DataXmlToDictProcessor(*args, process_namespaces: bool = False, namespace_separator: str = ':', namespaces: Optional[dict] = None, remove_namespaces: Optional[List[str]] = None, encoding: Optional[str] = None, item_depth: Optional[int] = None, strip_whitespace: bool = True, attr_prefix: str = '@', cdata_key: str = '#text', force_cdata: bool = False, cdata_separator: str = '', force_list: Optional[Any] = None, **kwargs)[source]¶: Bases: easydata.processors.data.DataBaseProcessor

8.1.7. DataTextFromReProcessor¶

class easydata.processors.data.DataTextFromReProcessor(*args, query, dotall=True, ignore_case=False, bytes_to_string_decode: str = 'utf-8', replace_keys: Optional[list] = None, none_if_empty=False, process_value=None, **kwargs)[source]¶: Bases: easydata.processors.data.DataBaseProcessor

8.1.8. DataJsonFromReToDictProcessor¶

class easydata.processors.data.DataJsonFromReToDictProcessor(*args, query, dotall=True, ignore_case=False, bytes_to_string_decode: str = 'utf-8', replace_keys: Optional[list] = None, none_if_empty=False, process_value=None, **kwargs)[source]¶: Bases: easydata.processors.data.DataTextFromReProcessor

8.1.9. DataFromQueryProcessor¶

class easydata.processors.data.DataFromQueryProcessor(query: Union[easydata.queries.base.QuerySearchBase, easydata.parsers.base.BaseData], **kwargs)[source]¶: Bases: easydata.processors.data.DataBaseProcessor

8.1.10. DataFromIterQueryProcessor¶

class easydata.processors.data.DataFromIterQueryProcessor(query: Union[easydata.queries.base.QuerySearchBase, easydata.parsers.base.BaseData], **kwargs)[source]¶: Bases: easydata.processors.data.DataFromQueryProcessor

8.1.11. DataVariantsProcessor¶

class easydata.processors.data.DataVariantsProcessor(query: Optional[easydata.queries.base.QuerySearchBase] = None, source: str = 'main', parser: Optional[easydata.parsers.base.BaseData] = None, key_parser: Optional[easydata.parsers.base.BaseData] = None, key_query: Optional[easydata.queries.base.QuerySearchBase] = None, **kwargs)[source]¶: Bases: easydata.processors.data.DataBaseProcessor