{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# What's new in RuleKit version 1.7.6?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 1. Manually initializing RuleKit is not longer necessary.\n", "\n", "Prior to this version, RuleKit had to be manually initialised using the `rulekit.RuleKit.init` method." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from rulekit import RuleKit\n", "from rulekit.classification import RuleClassifier\n", "\n", "RuleKit.init()\n", "\n", "clf = RuleClassifier()\n", "clf.fit(X, y)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now it is no longer necessary, and you can simply use any of the RuleKit operators directly." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from rulekit.classification import RuleClassifier\n", "\n", "clf = RuleClassifier()\n", "clf.fit(X, y)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 2. Introducing negated conditions for nominal attributes in rules.\n", "\n", "Using the new `complementary_conditions` parameter, the induction of negated conditions for nominal attributes can be enabled. Such conditions are of the form **attribute = !{value}**. This parameter has been added to all operator classes." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "IF stalk_surface_below_ring = !{y} AND spore_print_color = !{u} AND odor = !{n} AND stalk_root = !{c} THEN type = {p}\n", "IF bruises = {f} AND odor = !{n} THEN type = {p}\n", "IF stalk_surface_above_ring = {k} AND gill_spacing = {c} THEN type = {p}\n", "IF bruises = {f} AND stalk_surface_above_ring = !{f} AND stalk_surface_below_ring = !{f} AND ring_number = !{t} AND stalk_root = !{e} AND gill_attachment = {f} THEN type = {p}\n", "IF stalk_surface_below_ring = !{f} AND stalk_color_below_ring = !{n} AND spore_print_color = !{u} AND odor = !{a} AND gill_size = {n} AND cap_surface = !{f} THEN type = {p}\n", "IF cap_shape = !{s} AND cap_color = !{c} AND habitat = !{w} AND stalk_color_below_ring = !{g} AND stalk_surface_below_ring = !{y} AND spore_print_color = !{n} AND gill_spacing = {c} AND gill_color = !{u} AND stalk_root = !{c} AND stalk_color_above_ring = !{g} AND ring_type = !{f} AND veil_color = {w} THEN type = {p}\n", "IF cap_shape = !{c} AND stalk_surface_below_ring = !{y} AND spore_print_color = !{r} AND odor = {n} AND cap_surface = !{g} THEN type = {e}\n", "IF cap_color = !{y} AND cap_shape = !{c} AND stalk_color_below_ring = !{y} AND spore_print_color = !{r} AND odor = {n} AND cap_surface = !{g} THEN type = {e}\n", "IF spore_print_color = !{r} AND odor = !{f} AND stalk_color_above_ring = !{c} AND gill_size = {b} THEN type = {e}\n", "IF cap_color = !{p} AND cap_shape = !{c} AND habitat = !{u} AND stalk_color_below_ring = !{y} AND gill_color = !{b} AND spore_print_color = !{r} AND ring_number = !{n} AND odor = !{f} AND cap_surface = !{g} THEN type = {e}\n" ] } ], "source": [ "import pandas as pd\n", "from rulekit.classification import RuleClassifier\n", "\n", "df = pd.read_csv('https://raw.githubusercontent.com/stedy/Machine-Learning-with-R-datasets/master/mushrooms.csv')\n", "X = df.drop('type', axis=1)\n", "y = df['type']\n", "\n", "clf = RuleClassifier(complementary_conditions=True)\n", "clf.fit(X, y)\n", "\n", "for rule in clf.model.rules:\n", " print(rule)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 3. Approximate induction for classification rulesets.\n", "\n", "To reduce the training time on the classification datasets, the so-called *approximate induction* can now be used. This will force the algorithm not to check all possible numerical conditions during the rule induction phase. You can configure the number of bins you want to use as possible splits to limit the calculation.\n", "\n", "To enable *aproximate induction* use the `approximate_induction` parameter. To configure the maximum number of bins, use the `approximate_bins_count` parameter. At the moment, *aproximate induction* is only available for classification rule sets.\n", "\n", "The following example shows how using this function can reduce training time without sacrificing predictive accuracy." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
VariantTraining time [s]BAcc on test dataset
0Without approximate induction5.7300460.688744
1Without approximate induction0.1422590.703959
\n", "
" ], "text/plain": [ " Variant Training time [s] BAcc on test dataset\n", "0 Without approximate induction 5.730046 0.688744\n", "1 Without approximate induction 0.142259 0.703959" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import pandas as pd\n", "from rulekit.classification import RuleClassifier\n", "from sklearn.metrics import balanced_accuracy_score\n", "\n", "df_train = pd.read_parquet('https://github.com/cezary986/classification_tabular_datasets/raw/main/churn/train_test/train.parquet')\n", "df_test = pd.read_parquet('https://github.com/cezary986/classification_tabular_datasets/raw/main/churn/train_test/test.parquet')\n", "\n", "X_train = df_train.drop('class', axis=1)\n", "y_train = df_train['class']\n", "X_test = df_test.drop('class', axis=1)\n", "y_test = df_test['class']\n", "\n", "clf1 = RuleClassifier()\n", "clf2 = RuleClassifier(approximate_induction=True, approximate_bins_count=20)\n", "clf1.fit(X_train, y_train)\n", "clf2.fit(X_train, y_train)\n", "\n", "pd.DataFrame([\n", " {\n", " 'Variant': 'Without approximate induction',\n", " 'Training time [s]': clf1.model.total_time,\n", " 'BAcc on test dataset': balanced_accuracy_score(y_test, clf1.predict(X_test)),\n", " },\n", " {\n", " 'Variant': 'Without approximate induction',\n", " 'Training time [s]': clf2.model.total_time,\n", " 'BAcc on test dataset': balanced_accuracy_score(y_test, clf2.predict(X_test)),\n", " }\n", "])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 4. Observing and stopping the training process \n", "\n", "You can now watch the progress of the training process and stop it at a certain point. To do this, all you need to do is create a class extending the `events.RuleInductionProgressListener` class. Such a class should implement one of the following methods:\n", "* `on_new_rule(self, rule)`: Method called when the new rule was induced.\n", "* `on_progress(self, total_examples_count: int, uncovered_examples_count: int)`: Method that observes the training progress, how many examples have been covered relative to the total number of training examples. The division **uncovered_examples_count / total_examples_count** can be taken as some approximation of training progress. Keep in mind however that not in all scenarios the ruleset will cover all of the training examples. The increase in progress probably will not be linear.\n", "* `should_stop`: Method to stop the training process at a given point. If it returns **True**, the training will be stopped. You can then proceed to use the not fully trained model.\n", "\n", "Then you need to register your listener to the operator instance using `add_event_listener` method. All operators support this method.\n", "\n", "An example of the use of this mechanism is shown below." ] }, { "cell_type": "code", "execution_count": 36, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "IF number_customer_service_calls = (-inf, 3.50) AND account_length = (-inf, 224.50) AND total_day_minutes = (-inf, 239.95) AND international_plan = {no} THEN class = {no}\n", "IF number_customer_service_calls = (-inf, 3.50) AND total_day_minutes = (-inf, 254.05) AND international_plan = {no} THEN class = {no}\n", "IF number_customer_service_calls = (-inf, 3.50) AND total_day_minutes = (-inf, 255) AND international_plan = {no} THEN class = {no}\n", "IF number_customer_service_calls = (-inf, 3.50) AND total_day_minutes = (-inf, 263.25) AND international_plan = {no} THEN class = {no}\n", "IF total_intl_calls = (-inf, 19.50) AND number_customer_service_calls = (-inf, 3.50) AND total_eve_minutes = (-inf, 346.20) AND total_intl_minutes = (-inf, 19.85) AND total_day_calls = (-inf, 154) AND total_day_minutes = (-inf, 263.25) THEN class = {no}\n", "IF number_customer_service_calls = (-inf, 4.50) AND total_day_minutes = <1.30, 254.05) AND international_plan = {no} THEN class = {no}\n", "IF voice_mail_plan = {no} AND total_eve_minutes = <175.35, inf) AND total_day_calls = (-inf, 149) AND total_day_minutes = <263.25, inf) AND total_night_minutes = <115.85, inf) THEN class = {yes}\n" ] } ], "source": [ "import pandas as pd\n", "from rulekit.events import RuleInductionProgressListener\n", "from rulekit.rules import Rule\n", "from rulekit.classification import RuleClassifier\n", "\n", "df_train = pd.read_parquet('https://github.com/cezary986/classification_tabular_datasets/raw/main/churn/train_test/train.parquet')\n", "\n", "X_train = df_train.drop('class', axis=1)\n", "y_train = df_train['class']\n", "\n", "\n", "class MyProgressListener(RuleInductionProgressListener):\n", " _uncovered_examples_count: int = None\n", " _should_stop = False\n", "\n", " def on_new_rule(self, rule: Rule):\n", " pass\n", "\n", " def on_progress(\n", " self,\n", " total_examples_count: int,\n", " uncovered_examples_count: int\n", " ):\n", " if uncovered_examples_count < total_examples_count * 0.1:\n", " self._should_stop = True\n", "\n", " def should_stop(self) -> bool:\n", " return self._should_stop\n", " \n", "clf = RuleClassifier()\n", "clf.add_event_listener(MyProgressListener())\n", "clf.fit(X_train, y_train)\n", "\n", "for rule in clf.model.rules:\n", " print(rule)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To simplify the way learning is stopped after a certain number of rules have been learned, the parameter `max_rule_count` has been added to the operators. \n", "> This parameter in classification denotes the maximum number of rules for each possible class in the training dataset." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 5. Faster regression rules induction\n", "\n", "Prior to version 1.7.0, regression rule induction using this package was inherently slow due to the calculation of median values. It is now possible to use the new `mean_based_regression` parameter to enable faster regression using mean values instead of calculating median values. See the example below." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import pandas as pd\n", "from rulekit.regression import RuleRegressor\n", "\n", "df_train = pd.read_csv('./housing.csv')\n", "\n", "X_train = df_train.drop('class', axis=1)\n", "y_train = df_train['class']\n", "\n", "reg = RuleRegressor(mean_based_regression=True)\n", "reg.fit(X_train, y_train)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 5. Contrast set mining\n", "\n", "THe package now includes an algorithm for contrast set (CS) identification [Gudyś et al, 2022](https://doi.org/10.48550/arXiv.2204.00497).\n", "\n", "Following operator were introduced:\n", "\n", "* `classification.ContrastSetRuleClassifier`\n", "* `regression.ContrastSetRuleRegressor`\n", "* `survival.ContrastSetSurvivalRules`" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Other changes\n", "\n", "* New parameter `control_apriori_precision` added to the operators `classification.RuleClassifier` and `classification.ExpertRuleClassifier`. If enabled, checks if the candidate precision is higher than the apriori precision of the class under test during classification rule induction. Enabled by default.\n", "\n", "* Improved parameters types validation.\n", "\n", "* Parameter `minsupp_new` (alias `min_rule_covered`) now supports floating point values - the same as in original RuleKit package.\n", "\n", "* Fix bug when models were fitted or applied to DataFrames with boolean columns. (Issue [#18](https://github.com/adaa-polsl/RuleKit-python/issues/18))\n", "\n", "* Fixed Issue [#10](https://github.com/adaa-polsl/RuleKit-python/issues/10) - *predict_proba method does not work for ExpertRuleClassifier*\n", "\n", "* Update dependencies versions." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Deprecations\n", "\n", "* Renamed the `min_rule_covered` parameter to `minsupp_new`. The old name is now deprecated, but will be treated as an alias to the next major version, resulting in a warning only." ] } ], "metadata": { "kernelspec": { "display_name": "env", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.10" } }, "nbformat": 4, "nbformat_minor": 2 }