Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add functional API for algorithm contributtions #876

Open
wants to merge 33 commits into
base: develop
Choose a base branch
from

Conversation

Czaki
Copy link
Collaborator

@Czaki Czaki commented Jan 3, 2023

Summary by CodeRabbit

  • New Features

    • Enhanced the AlgorithmDescribeBaseMeta class to support additional parameters and methods for class creation from functions.
    • Added new class attributes and static methods to improve algorithm description and instantiation.
  • Enhancements

    • Improved documentation with docstrings for need_segmentation and need_mask methods.
    • Refactored get_extensions method for better readability and efficiency.
    • Updated load_metadata_base function to provide clearer error handling.
  • Bug Fixes

    • Fixed issues with class definitions in BaseSmoothing, NoiseFilteringBase, BaseThreshold, and BaseWatershed to correctly incorporate new method attributes.
  • Documentation

    • Added comprehensive docstrings and comments to clarify the purpose and usage of new methods and classes.
  • Tests

    • Updated test cases to reflect changes in data types and type checking.
    • Added new test cases for algorithm description, utility functions, and thresholding methods to ensure robustness and correctness.

@Czaki Czaki added this to the 0.15.0 milestone Jan 3, 2023
@sourcery-ai
Copy link
Contributor

sourcery-ai bot commented Jan 3, 2023

Sourcery Code Quality Report

❌  Merging this PR will decrease code quality in the affected files by 0.92%.

Quality metrics Before After Change
Complexity 2.77 ⭐ 3.05 ⭐ 0.28 👎
Method Length 47.27 ⭐ 49.39 ⭐ 2.12 👎
Working memory 6.50 🙂 6.09 ⭐ -0.41 👍
Quality 77.22% 76.30% -0.92% 👎
Other metrics Before After Change
Lines 2251 2625 374
Changed files Quality Before Quality After Quality Change
package/PartSegCore/algorithm_describe_base.py 66.67% 🙂 65.14% 🙂 -1.53% 👎
package/PartSegCore/io_utils.py 83.15% ⭐ 83.08% ⭐ -0.07% 👎
package/PartSegCore/segmentation/algorithm_base.py 81.07% ⭐ 81.07% ⭐ 0.00%
package/PartSegCore/segmentation/border_smoothing.py 80.96% ⭐ 80.96% ⭐ 0.00%
package/PartSegCore/segmentation/noise_filtering.py 92.16% ⭐ 92.05% ⭐ -0.11% 👎
package/PartSegCore/segmentation/threshold.py 80.98% ⭐ 80.30% ⭐ -0.68% 👎
package/PartSegCore/segmentation/watershed.py 72.55% 🙂 72.55% 🙂 0.00%
package/tests/test_PartSegCore/test_algorithm_describe_base.py 78.51% ⭐ 79.88% ⭐ 1.37% 👍
package/tests/test_PartSegCore/segmentation/test_threshold.py 67.20% 🙂 66.39% 🙂 -0.81% 👎

Here are some functions in these files that still need a tune-up:

File Function Complexity Length Working Memory Quality Recommendation
package/PartSegCore/algorithm_describe_base.py ROIExtractionProfile._pretty_print 21 😞 224 ⛔ 12 😞 34.52% 😞 Refactor to reduce nesting. Try splitting into smaller methods. Extract out complex expressions
package/PartSegCore/algorithm_describe_base.py AlgorithmDescribeBaseMeta.from_function 15 🙂 236 ⛔ 38.35% 😞 Try splitting into smaller methods
package/PartSegCore/segmentation/watershed.py MSOWatershed.sprawl 6 ⭐ 194 😞 14 😞 46.32% 😞 Try splitting into smaller methods. Extract out complex expressions
package/PartSegCore/algorithm_describe_base.py AlgorithmProperty.__init__ 8 ⭐ 163 😞 14 😞 47.14% 😞 Try splitting into smaller methods. Extract out complex expressions
package/PartSegCore/algorithm_describe_base.py AlgorithmDescribeBaseMeta._validate_function_parameters 9 🙂 141 😞 15 😞 47.24% 😞 Try splitting into smaller methods. Extract out complex expressions

Legend and Explanation

The emojis denote the absolute quality of the code:

  • ⭐ excellent
  • 🙂 good
  • 😞 poor
  • ⛔ very poor

The 👍 and 👎 indicate whether the quality has improved or gotten worse with this pull request.


Please see our documentation here for details on how these metrics are calculated.

We are actively working on this report - lots more documentation and extra metrics to come!

Help us improve this quality report!

@deprecated-napari-hub-preview-bot
Copy link

deprecated-napari-hub-preview-bot bot commented Jan 3, 2023

Preview page for your plugin is ready here:
https://preview.napari-hub.org/4DNucleome/PartSeg/876
Updated: 2023-01-11T18:54:45.526162

@codecov
Copy link

codecov bot commented Jan 3, 2023

Codecov Report

Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (de43a4f) 87.84% compared to head (ac7e352) 90.88%.

❗ Current head ac7e352 differs from pull request most recent head 28b697a. Consider uploading reports for the commit 28b697a to get more accurate results

Files Patch % Lines
package/PartSegCore/io_utils.py 75.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #876      +/-   ##
===========================================
+ Coverage    87.84%   90.88%   +3.03%     
===========================================
  Files          151      196      +45     
  Lines        20484    30464    +9980     
===========================================
+ Hits         17995    27688    +9693     
- Misses        2489     2776     +287     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Czaki Czaki changed the title Add functional api for plugins feat: Add functional API for algorithm contributtions Jan 4, 2023
@sonarcloud
Copy link

sonarcloud bot commented Jan 11, 2023

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

@sonarcloud
Copy link

sonarcloud bot commented May 18, 2023

SonarCloud Quality Gate failed.    Quality Gate failed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot E 1 Security Hotspot
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

@Czaki Czaki modified the milestones: 0.15.0, 0.15 May 22, 2023
Copy link
Contributor

coderabbitai bot commented Dec 4, 2023

Walkthrough

The changes across various files reflect enhancements in a Python codebase, particularly in a CI/CD pipeline and algorithmic class definitions. New CI steps, class attributes, static methods, and decorators have been introduced, alongside improved error handling and type checking. These updates aim to streamline development workflows, enforce better coding practices, and extend functionality within the codebase.

Changes

File Path Change Summary
.github/workflows/... Added "Upload coverage" job to CI/CD pipeline.
package/PartSegCore/algorithm_describe_base.py
package/PartSegCore/io_utils.py
Updated AlgorithmDescribeBaseMeta with new methods and attributes.
Enhanced docstrings, refactored get_extensions, and modified exception handling in load_metadata_base.
package/PartSegCore/segmentation/... Added method_from_fun parameter to class definitions in border_smoothing.py, noise_filtering.py, threshold.py, and watershed.py.
package/PartSegCore/utils.py Introduced kwargs_to_model decorator for backward compatibility.
package/tests/test_PartSegCore/segmentation/test_threshold.py
package/tests/test_PartSegCore/test_algorithm_describe_base.py
package/tests/test_PartSegCore/test_utils.py
Specified data type for arrays and added type checking.
Added new classes and tests for class generation from functions.
Added kwargs_to_model function and related test.

"In the burrow of the code, changes abound,
🐇 Hopping through the lines, improvements found.
With each commit push, and pipeline flow,
A rabbit's touch, makes the garden grow."

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share

Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit-tests for this file.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit tests for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository from git and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit tests.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger a review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • The JSON schema for the configuration file is available here.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/coderabbit-overrides.v2.json

CodeRabbit Discord Community

Join our Discord Community to get help, request features, and share feedback.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 2

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between fd6e1b8 and d555168.
Files ignored due to filter (1)
  • setup.cfg
Files selected for processing (11)
  • .github/workflows/base_test_workflow.yml (1 hunks)
  • package/PartSegCore/algorithm_describe_base.py (5 hunks)
  • package/PartSegCore/io_utils.py (3 hunks)
  • package/PartSegCore/segmentation/border_smoothing.py (1 hunks)
  • package/PartSegCore/segmentation/noise_filtering.py (2 hunks)
  • package/PartSegCore/segmentation/threshold.py (4 hunks)
  • package/PartSegCore/segmentation/watershed.py (1 hunks)
  • package/PartSegCore/utils.py (2 hunks)
  • package/tests/test_PartSegCore/segmentation/test_threshold.py (3 hunks)
  • package/tests/test_PartSegCore/test_algorithm_describe_base.py (4 hunks)
  • package/tests/test_PartSegCore/test_utils.py (2 hunks)
Additional comments: 37
.github/workflows/base_test_workflow.yml (1)
  • 115-120: The new "Upload coverage" step has been added to the workflow, which aligns with the PR objectives to enhance the CI/CD pipeline. However, ensure that the coverage.xml file is generated before this step is executed, as the hunk does not show the generation of this file.
package/PartSegCore/algorithm_describe_base.py (5)
  • 16-25: The addition of imports and the AlgorithmDescribeNotFound exception class are correctly implemented and documented.

  • 120-126: > Note: This review was outside the patches, so it was mapped to the patch with the greatest overlap. Original lines [122-130]

The changes to the AlgorithmDescribeBaseMeta class's __new__ method correctly handle the new parameters and set class attributes based on the presence of abstract methods.

  • 150-298: The new static methods and the updated from_function method in the AlgorithmDescribeBaseMeta class are correctly implemented to support the creation of classes from functions.

  • 307-317: The AlgorithmDescribeBase class has been correctly updated with a new __new__ method and a get_doc_from_fields method to support class creation from functions and generate documentation.

  • 328-352: The from_function methods in the AlgorithmDescribeBase class are correctly overloaded to handle different parameter sets for class generation from functions.

package/PartSegCore/io_utils.py (4)
  • 103-111: Docstrings added to need_segmentation and need_mask methods improve code documentation.

  • 113-113: The summary mentions refactoring of get_extensions method using the walrus operator, but the provided hunk does not show this change.

  • 164-169: Refactoring of get_extensions method using the walrus operator in LoadBase class enhances readability.

  • 206-211: Modification in load_metadata_base to suppress the original exception could impact error handling and debugging.

package/PartSegCore/segmentation/border_smoothing.py (3)
  • 14-14: The addition of method_from_fun="smooth" to the BaseSmoothing class definition is consistent with the PR objectives to introduce a functional API for algorithm contributions. This change should be verified to ensure that it integrates correctly with the rest of the functional API infrastructure.

  • 15-15: The __argument_class__ attribute is set to BaseModel in the BaseSmoothing class. Ensure that all subclasses of BaseSmoothing that require specific argument models override this attribute accordingly.

  • 17-17: The smooth method in BaseSmoothing is correctly defined as an abstract class method, which enforces that all subclasses must provide their own implementation. This is a good practice for abstract base classes.

package/PartSegCore/segmentation/noise_filtering.py (3)
  • 3-3: The addition of the abstractmethod import is correct and necessary for the changes made to the NoiseFilteringBase class.

  • 29-30: The use of @abstractmethod decorator on noise_filter is appropriate to ensure that subclasses implement this method.

  • 32-32: The subclasses of NoiseFilteringBase correctly implement the noise_filter method, adhering to the contract established by the abstract base class.

package/PartSegCore/segmentation/threshold.py (4)
  • 2-8: The import of mahotas is correctly added and used within the riddler_calvard function.

  • 40-40: The BaseThreshold class now inherits from AlgorithmDescribeBase and ABC, which aligns with the PR objective to support algorithm contributions through a functional API.

  • 279-280: The MahotasThreshold class does not inherit from any base class. This seems inconsistent with the design of other threshold classes in the file, which typically inherit from BaseThreshold or another base class. Please verify if this is intentional or an oversight.

  • 283-323: The riddler_calvard function is correctly implemented and registered in ThresholdSelection, which is in line with the PR objectives.

package/PartSegCore/segmentation/watershed.py (1)
  • 25-25: The summary mentions the addition of method_from_fun as a keyword argument in the BaseWatershed class definition, but the hunk shows it as a class attribute. This should be corrected in the summary or the code, depending on the intended implementation.
package/PartSegCore/utils.py (6)
  • 7-13: The addition of new imports seems appropriate for the new functionality introduced in the file.

  • 468-546: The implementation of the kwargs_to_model decorator and its helper functions appears to be correct and follows best practices for preserving function metadata and providing backward compatibility.

  • 468-484: The _get_translation_dicts_from_signature function correctly extracts translation dictionaries from the function signature and ensures that the parameters have BaseModel type annotations.

  • 487-493: The _get_kwargs_from_old_args function properly maps old positional arguments to keyword arguments based on the provided old_args_order list.

  • 496-505: The _map_kwargs_to_model function correctly maps keyword arguments to model instances, ensuring that each keyword argument corresponds to a field in the model.

  • 508-546: The kwargs_to_model decorator is well-implemented, with clear error handling and a warning for deprecated usage. It ensures backward compatibility while encouraging users to update their function calls.

package/tests/test_PartSegCore/segmentation/test_threshold.py (3)
  • 15-21: The explicit specification of dtype=np.uint32 for square and cube arrays ensures that the data type is consistent and matches the expected type for the tests.

  • 42-42: The addition of type checking for thr_info to ensure it is an integer, float, or np.integer is a good practice for data type validation in tests.

  • 55-56: The type checking for the elements of thr_info in test_double_threshold function ensures that both thresholds returned are of the correct type, which is a good practice for robust testing.

package/tests/test_PartSegCore/test_algorithm_describe_base.py (5)
  • 1-6: The addition of ABC and abstractmethod imports suggests that abstract base classes and methods are being used, which is consistent with the PR's objective to enhance the codebase for algorithm contributions.

  • 10-16: The import statements are correctly formatted and relevant to the PR's objectives.

  • 519-535: The introduction of ClassForTestFromFuncBase and ClassForTestFromFunc with abstract methods aligns with the PR's objective to provide a structured way for developers to contribute new algorithms.

  • 359-493: The new test cases added are comprehensive and appear to cover the functionality related to class generation from functions, which is a key part of the PR's objectives.

  • 496-502: The test case for ROIExtractionProfile is consistent with the PR's objectives and ensures that the new functional API for algorithm contributions is properly tested.

package/tests/test_PartSegCore/test_utils.py (2)
  • 13-18: The addition of kwargs_to_model to the import list is consistent with the PR objectives and the summary provided.

  • 367-395: The new test function test_kwargs_to_model is well-structured and covers various scenarios to ensure the kwargs_to_model decorator works as expected, including correct usage, deprecation warnings, and errors for incorrect usage.

Comment on lines 119 to 120
path: ./coverage.xml
retention-days: 5
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The retention period for the coverage.xml artifact is set to 5 days, which is different from the 7-day retention period for other artifacts. Confirm if this inconsistency is intentional.

Copy link

sonarcloud bot commented Dec 4, 2023

SonarCloud Quality Gate failed.    Quality Gate failed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot E 1 Security Hotspot
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

idea Catch issues before they fail your Quality Gate with our IDE extension sonarlint SonarLint

Copy link

sonarcloud bot commented Jan 26, 2024

Quality Gate Failed Quality Gate failed

Failed conditions

1 Security Hotspot

See analysis details on SonarCloud

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 9

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between d555168 and 28b697a.
Files ignored due to path filters (1)
  • setup.cfg is excluded by: !**/*.cfg
Files selected for processing (4)
  • package/PartSegCore/algorithm_describe_base.py (5 hunks)
  • package/PartSegCore/io_utils.py (5 hunks)
  • package/PartSegCore/segmentation/algorithm_base.py (1 hunks)
  • package/tests/test_PartSegCore/test_algorithm_describe_base.py (4 hunks)
Additional comments: 20
package/PartSegCore/segmentation/algorithm_base.py (2)
  • 207-207: The method get_info_text now correctly specifies a return type of str, aligning with best practices for type hinting in Python.
  • 207-207: Ensure the calculation_run method correctly specifies its return type as ROIExtractionResult as per the summary. This change enhances type safety and code readability.
Verification successful

The calculation_run method correctly specifies its return type as ROIExtractionResult in multiple instances within the PartSegCore package, as indicated by the search results. This confirms the implementation aligns with the summary's description, enhancing type safety and code readability.

Scripts Executed

The following scripts were executed for the analysis:


Script:

#!/bin/bash
# Verify the return type of calculation_run method
ast-grep --lang python --pattern $'def calculation_run($_, $_) -> ROIExtractionResult { $$$ }'

Length of output: 440



Script:

#!/bin/bash
# Adjusting the pattern to find the calculation_run method without specifying parameters or return type
ast-grep --lang python --pattern $'def calculation_run($$$) { $$$ }'

Length of output: 524



Script:

#!/bin/bash
# Using ripgrep to search for the calculation_run method in Python files
rg "def calculation_run" --type py

Length of output: 2491

package/PartSegCore/io_utils.py (4)
  • 71-71: Adding a return type of str to the get_short_name method in SaveBase and LoadBase classes improves type clarity and consistency.
  • 105-105: The addition of documentation for the need_segmentation method clarifies its purpose and usage, enhancing code readability.
  • 110-110: Similarly, adding documentation to the need_mask method improves understanding of its necessity in certain save operations.
  • 166-166: The use of the walrus operator in get_extensions method simplifies the code and improves readability by avoiding an extra line of code for the match operation.
package/tests/test_PartSegCore/test_algorithm_describe_base.py (14)
  • 3-3: The import of ABC and abstractmethod is necessary for defining abstract base classes and methods, aligning with Python's way of enforcing class interfaces.
  • 13-13: The introduction of AlgorithmDescribeBaseMeta in the imports suggests enhancements to metaclass functionalities, potentially for dynamic class creation or attribute enforcement.
  • 359-377: The tests for generating classes from functions validate the functionality and error handling of the from_function method, ensuring robustness and correct behavior.
  • 379-385: The test for missing return annotation correctly checks for runtime errors when abstract methods in subclasses do not specify return types, enforcing good coding practices.
  • 387-398: This test validates that the from_function method raises an appropriate error when used with classes that do not support it, ensuring that the method's usage is restricted to compatible classes.
  • 400-405: The test for passing incorrect types to the from_function method ensures type safety and correct parameter handling, contributing to the robustness of the class generation feature.
  • 407-424: The comprehensive test for the from_function method demonstrates its capability to dynamically create classes from functions, validating the feature's core functionality.
  • 426-435: Testing the from_function decorator without explicit parameters ensures flexibility and ease of use in class generation, highlighting the feature's adaptability.
  • 437-442: This test checks for errors when additional, undefined parameters are passed to the from_function method, ensuring strict parameter validation.
  • 444-449: The test for positional-only arguments in the from_function method ensures compatibility with Python's function signature restrictions, maintaining consistency in method definitions.
  • 451-472: Testing the from_function method as a decorator on a class with abstract methods validates its versatility and integration with Python's class system.
  • 474-485: The test for class generation without user-provided attributes through the from_function decorator demonstrates the method's ability to handle minimalistic function definitions.
  • 487-493: Testing the handling of keyword arguments in functions used with from_function ensures that dynamically generated classes can accept and process variable arguments.
  • 496-500: The test for ROIExtractionProfile instantiation and the use of pytest.warns to check for deprecation warnings ensures backward compatibility and future-proofing of the API.

Comment on lines 129 to 309
"""
Validate if all parameters without default values are defined in self.__calculation_method__

:param func: function to validate
:return: set of parameters that should be dropped
"""
signature = inspect.signature(func)
base_method_signature = inspect.signature(method)
take_all = False

for parameter in signature.parameters.values():
if parameter.kind in {inspect.Parameter.VAR_POSITIONAL, inspect.Parameter.POSITIONAL_ONLY}:
raise ValueError(f"Function {func} should not have positional only parameters")
if (
parameter.default is inspect.Parameter.empty
and parameter.name not in base_method_signature.parameters
and parameter.kind != inspect.Parameter.VAR_KEYWORD
):
raise ValueError(f"Parameter {parameter.name} is not defined in {method_name} method")

if parameter.kind == inspect.Parameter.VAR_KEYWORD:
take_all = True

if take_all:
return set()

return {
parameters.name
for parameters in base_method_signature.parameters.values()
if parameters.name not in signature.parameters
}

@staticmethod
def _get_argument_class_from_signature(func, argument_name: str):
signature = inspect.signature(func)
if argument_name not in signature.parameters:
return BaseModel
return signature.parameters[argument_name].annotation

@staticmethod
def _get_parameters_from_signature(func):
signature = inspect.signature(func)
return [parameters.name for parameters in signature.parameters.values()]

def from_function(self, func=None, **kwargs):
"""generate new class from function"""

# Test if all abstract methods values are provided in kwargs

if not self.__support_from_function__:
raise RuntimeError("This class does not support from_function method")

self._validate_if_all_abstract_getters_are_defined(self.__abstract_getters__, kwargs)

# check if all values have correct type
for key, value in kwargs.items():
if not isinstance(value, self.__abstract_getters__[key]):
raise TypeError(f"Value for {key} should be {self.__abstract_getters__[key]}")

def _getter_by_name(name):
def _func():
return kwargs[name]

return _func

parameters_order = self._get_parameters_from_signature(getattr(self, self.__method_name__))

def _class_generator(func_):
drop_attr = self._validate_function_parameters(
func_, getattr(self, self.__method_name__), self.__method_name__
)

@wraps(func_)
def _calculate_method(*args, **kwargs_):
for attr, name in zip(args, parameters_order):
if name in kwargs_:
raise ValueError(f"Parameter {name} is defined twice")
kwargs_[name] = attr

for name in drop_attr:
kwargs_.pop(name, None)
return func_(**kwargs_)

class_dkt = {f"get_{name}": _getter_by_name(name) for name in self.__abstract_getters__}

class_dkt[self.__method_name__] = _calculate_method
class_dkt["__argument_class__"] = self._get_argument_class_from_signature(
func_, self.__additional_parameters_name__
)
class_dkt["__from_function__"] = True

return type(func_.__name__.replace("_", " ").title().replace(" ", ""), (self,), class_dkt)

if func is None:
return _class_generator
return _class_generator(func)


class AlgorithmDescribeBase(ABC, metaclass=AlgorithmDescribeBaseMeta):
"""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 NOTE
This review was outside the diff hunks and was mapped to the diff hunk with the greatest overlap. Original lines [123-147]

The modifications to the AlgorithmDescribeBaseMeta metaclass introduce new parameters method_from_fun and additional_parameters to the __new__ method. These changes are designed to support the creation of classes from functions, enhancing the flexibility of algorithm definitions. However, there's a potential issue with the logic that determines whether a class supports the from_function method based on the presence of __method_name__ and __additional_parameters_name__. This logic may not accurately reflect all scenarios where from_function could be applicable or beneficial. Additionally, the error handling in the absence of __argument_class__ or get_fields functions could be more descriptive, providing guidance on how to resolve the issue.

Consider refining the logic that determines the support for from_function to cover more scenarios and improve the error message when __argument_class__ or get_fields functions are missing to guide developers on how to make their classes compatible.

Comment on lines +150 to +173
@staticmethod
def _get_abstract_getters(
cls2, support_from_function, calculation_method
) -> typing.Tuple[typing.Dict[str, typing.Any], bool]:
abstract_getters = {}
if hasattr(cls2, "__abstractmethods__") and cls2.__abstractmethods__:
# get all abstract methods that starts with `get_`
for method_name in cls2.__abstractmethods__:
if method_name.startswith("get_"):
method = getattr(cls2, method_name)
if "return" not in method.__annotations__:
msg = f"Method {method_name} of {cls2.__qualname__} need to have return type defined"
try:
file_name = inspect.getsourcefile(method)
line = inspect.getsourcelines(method)[1]
msg += f" in {file_name}:{line}"
except TypeError:
pass
raise RuntimeError(msg)

abstract_getters[method_name[4:]] = getattr(cls2, method_name).__annotations__["return"]
elif method_name != calculation_method:
support_from_function = False
return abstract_getters, support_from_function
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The _get_abstract_getters method has been modified to support the new functionality related to creating classes from functions. This method now checks for abstract methods starting with get_ and validates their return type annotations. While this approach ensures that abstract getters are properly defined, it may be too restrictive by excluding potentially valid abstract methods that do not follow this naming convention. Additionally, the error message generated when a return type is not defined could be enhanced to provide more specific guidance on how to correct the issue.

Consider allowing more flexibility in the naming convention of abstract methods and improve the error message for missing return type annotations to offer clearer guidance on resolving the issue.

Comment on lines +175 to +186
@staticmethod
def _get_calculation_method_params_name(cls2) -> typing.Optional[str]:
if cls2.__method_name__ is None:
return None
signature = inspect.signature(getattr(cls2, cls2.__method_name__))
if "arguments" in signature.parameters:
return "arguments"
if "params" in signature.parameters:
return "params"
if "parameters" in signature.parameters:
return "parameters"
raise RuntimeError(f"Cannot determine arguments parameter name in {cls2.__method_name__}")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The _get_calculation_method_params_name method attempts to determine the name of the parameter that represents arguments in the calculation method. This method assumes specific parameter names (arguments, params, parameters) and raises a runtime error if none are found. This approach may not accommodate all valid parameter naming conventions used by developers, potentially limiting the flexibility of the API.

Enhance the method to support a wider range of parameter naming conventions or provide a mechanism for developers to specify the parameter name explicitly, improving the API's flexibility.

Comment on lines +188 to +206
@staticmethod
def _validate_if_all_abstract_getters_are_defined(abstract_getters, kwargs):
abstract_getters_set = set(abstract_getters)
kwargs_set = set(kwargs.keys())

if abstract_getters_set != kwargs_set:
# Provide a nice error message with information about what is missing and is obsolete
missing_text = ", ".join(sorted(abstract_getters_set - kwargs_set))
if missing_text:
missing_text = f"Not all abstract methods are provided, missing: {missing_text}."
else:
missing_text = ""
extra_text = ", ".join(sorted(kwargs_set - abstract_getters_set))
if extra_text:
extra_text = f"There are extra attributes in call: {extra_text}."
else:
extra_text = ""

raise ValueError(f"{missing_text} {extra_text}")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The _validate_if_all_abstract_getters_are_defined method checks if all required abstract getters are provided in kwargs. While this validation is crucial for ensuring that all necessary information is provided, the error message generated when there are missing or extra attributes could be more actionable. Specifically, it could guide developers on how to correct the issue by providing examples or suggesting where to define the missing getters.

Improve the error message generated by _validate_if_all_abstract_getters_are_defined to include actionable guidance on how to resolve issues with missing or extra attributes.

Comment on lines +208 to +238
@staticmethod
def _validate_function_parameters(func, method, method_name) -> set:
"""
Validate if all parameters without default values are defined in self.__calculation_method__

:param func: function to validate
:return: set of parameters that should be dropped
"""
signature = inspect.signature(func)
base_method_signature = inspect.signature(method)
take_all = False

for parameter in signature.parameters.values():
if parameter.kind in {inspect.Parameter.VAR_POSITIONAL, inspect.Parameter.POSITIONAL_ONLY}:
raise ValueError(f"Function {func} should not have positional only parameters")
if (
parameter.default is inspect.Parameter.empty
and parameter.name not in base_method_signature.parameters
and parameter.kind != inspect.Parameter.VAR_KEYWORD
):
raise ValueError(f"Parameter {parameter.name} is not defined in {method_name} method")

if parameter.kind == inspect.Parameter.VAR_KEYWORD:
take_all = True

if take_all:
return set()

return {
parameters.name
for parameters in base_method_signature.parameters.values()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The _validate_function_parameters method validates if all parameters without default values are defined in the calculation method. This validation is essential for ensuring compatibility between the function and the class it is being converted into. However, the method currently raises a generic ValueError for missing parameters, which may not provide enough context for developers to understand the source of the error and how to fix it.

Enhance the error handling in _validate_function_parameters to provide more specific and actionable error messages, helping developers identify and correct issues with missing parameters more efficiently.

Comment on lines +254 to +305
def from_function(self, func=None, **kwargs):
"""generate new class from function"""

# Test if all abstract methods values are provided in kwargs

if not self.__support_from_function__:
raise RuntimeError("This class does not support from_function method")

self._validate_if_all_abstract_getters_are_defined(self.__abstract_getters__, kwargs)

# check if all values have correct type
for key, value in kwargs.items():
if not isinstance(value, self.__abstract_getters__[key]):
raise TypeError(f"Value for {key} should be {self.__abstract_getters__[key]}")

def _getter_by_name(name):
def _func():
return kwargs[name]

return _func

parameters_order = self._get_parameters_from_signature(getattr(self, self.__method_name__))

def _class_generator(func_):
drop_attr = self._validate_function_parameters(
func_, getattr(self, self.__method_name__), self.__method_name__
)

@wraps(func_)
def _calculate_method(*args, **kwargs_):
for attr, name in zip(args, parameters_order):
if name in kwargs_:
raise ValueError(f"Parameter {name} is defined twice")
kwargs_[name] = attr

for name in drop_attr:
kwargs_.pop(name, None)
return func_(**kwargs_)

class_dkt = {f"get_{name}": _getter_by_name(name) for name in self.__abstract_getters__}

class_dkt[self.__method_name__] = _calculate_method
class_dkt["__argument_class__"] = self._get_argument_class_from_signature(
func_, self.__additional_parameters_name__
)
class_dkt["__from_function__"] = True

return type(func_.__name__.replace("_", " ").title().replace(" ", ""), (self,), class_dkt)

if func is None:
return _class_generator
return _class_generator(func)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The from_function method in the AlgorithmDescribeBaseMeta class is a significant addition, enabling the generation of new classes from functions. This method includes comprehensive validation and error handling to ensure that the function and provided arguments meet the necessary criteria. However, the method's complexity and the multiple nested functions could impact readability and maintainability. Additionally, the method assumes that the function's name follows a specific naming convention (func_.__name__.replace("_", " ").title().replace(" ", "")), which may not always be desirable or applicable.

Consider refactoring the from_function method to simplify its structure and improve readability. Additionally, provide a way for developers to specify the class name explicitly when it cannot be appropriately inferred from the function's name.

Comment on lines +318 to +321
def __new__(cls, *args, **kwargs):
if cls.__from_function__:
return getattr(cls, cls.__method_name__)(*args, **kwargs)
return super().__new__(cls)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The __new__ method in the AlgorithmDescribeBase class has been modified to support instances created from functions. This modification allows for dynamic behavior based on whether the class was generated from a function. While this approach adds flexibility, it also introduces complexity into the instantiation process, potentially affecting the predictability and debuggability of classes derived from AlgorithmDescribeBase.

Evaluate the impact of the modified __new__ method on the predictability and debuggability of classes derived from AlgorithmDescribeBase. Consider providing additional documentation or examples to help developers understand and effectively use this new functionality.

Comment on lines +345 to +356
@classmethod
def from_function(
cls: TypeT, func=None, **kwargs
) -> typing.Union[TypeT, typing.Callable[[typing.Callable], TypeT]]:
def _from_function(func_) -> typing.Type["AlgorithmDescribeBase"]:
if "name" not in kwargs:
kwargs["name"] = func_.__name__.replace("_", " ").title()
return AlgorithmDescribeBaseMeta.from_function(cls, func_, **kwargs)

if func is None:
return _from_function
return _from_function(func)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The from_function class method in the AlgorithmDescribeBase class provides an overload mechanism to support different usage patterns (with or without a specified function). This method leverages the AlgorithmDescribeBaseMeta.from_function to generate new classes from functions, demonstrating a clear separation of concerns between the metaclass and the class itself. However, the documentation could be improved to better explain the method's overloads and provide examples of how to use each pattern effectively.

Enhance the documentation for the from_function class method in AlgorithmDescribeBase to include examples and explanations of the different usage patterns supported by the method's overloads.

@@ -205,7 +206,7 @@ def load_metadata_base(data: typing.Union[str, Path]):
try:
decoded_data = json.loads(str(data), object_hook=partseg_object_hook)
except Exception: # pragma: no cover
raise e # noqa: B904
raise e from None
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Modifying exception handling in load_metadata_base to use raise e from None suppresses the context of the original exception, which might hinder debugging. Consider if this change aligns with the desired error handling behavior.

-            raise e from None
+            raise e

Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.

Suggested change
raise e from None
raise e

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant