Automa-通過(guò)連接塊來(lái)自動(dòng)化你的瀏覽器
1、前言通過(guò)瀏覽器插件可實(shí)現(xiàn)自動(dòng)化腳本的錄制與編寫(xiě)，具有代表性的工具就是：Selenium IDE、Katalon Recorder，對(duì)于簡(jiǎn)單的業(yè)務(wù)來(lái)說(shuō)可快速實(shí)現(xiàn)自動(dòng)化的上手工作。Selenium IDEKat
作為數(shù)據(jù)科學(xué)家，我們常常使用 Jupyter Notebooks 進(jìn)行數(shù)據(jù)探索和模型開(kāi)發(fā)。在這個(gè)階段，我們關(guān)注的重點(diǎn)是快速驗(yàn)證想法和證明概念。然而，一旦模型準(zhǔn)備就緒，就需要將其部署到生產(chǎn)環(huán)境中，這時(shí)代碼質(zhì)量就顯得尤為重要。
生產(chǎn)代碼必須足夠健壯、可讀且易于維護(hù)。不幸的是，數(shù)據(jù)科學(xué)家編寫(xiě)的原型代碼通常難以滿(mǎn)足這些要求。作為一名機(jī)器學(xué)習(xí)工程師，我的職責(zé)就是確保代碼能夠順利地從概念驗(yàn)證階段過(guò)渡到生產(chǎn)環(huán)境。
因此，編寫(xiě)簡(jiǎn)潔的代碼對(duì)于提高開(kāi)發(fā)效率和降低維護(hù)成本至關(guān)重要。在本文中，我將分享一些 Python 編程技巧和最佳實(shí)踐，并通過(guò)簡(jiǎn)潔的代碼示例，向您展示如何提高代碼的可讀性和可維護(hù)性。
我衷心希望這篇文章能為 Python 愛(ài)好者提供有價(jià)值的見(jiàn)解，特別是能夠激勵(lì)更多的數(shù)據(jù)科學(xué)家重視代碼質(zhì)量，因?yàn)楦哔|(zhì)量的代碼不僅有利于開(kāi)發(fā)過(guò)程，更能確保模型成功地投入生產(chǎn)使用。
很多開(kāi)發(fā)人員沒(méi)有遵循給變量和函數(shù)命名富有意義的名稱(chēng)這一最佳實(shí)踐。代碼的可讀性和可維護(hù)性因此大大降低。
命名對(duì)于代碼質(zhì)量至關(guān)重要。好的命名不僅能直觀(guān)地表達(dá)代碼的功能，而且還能避免過(guò)多的注釋和解釋?zhuān)岣叽a的整潔度。一個(gè)描述性強(qiáng)的名稱(chēng)，就能讓函數(shù)的作用一目了然。
你給出的機(jī)器學(xué)習(xí)例子非常恰當(dāng)。比如加載數(shù)據(jù)集并將其分割為訓(xùn)練集和測(cè)試集這一常見(jiàn)任務(wù),如果使用富有意義的函數(shù)名如load_dataset()和split_into_train_test()就能立刻看出這兩個(gè)函數(shù)的用途，而不需要查閱注釋。
可讀性強(qiáng)的代碼不僅能讓其他開(kāi)發(fā)者更快理解，自己在未來(lái)維護(hù)時(shí)也能事半功倍。因此，我們應(yīng)當(dāng)養(yǎng)成良好的命名習(xí)慣，寫(xiě)出簡(jiǎn)潔直白的代碼。
以一個(gè)典型的機(jī)器學(xué)習(xí)例子為例：加載數(shù)據(jù)集并將其分割成訓(xùn)練集和測(cè)試集。
import pandas as pdfrom sklearn.model_selection import train_test_splitdef load_and_split(d): df = pd.read_csv(d) X = df.iloc[:, :-1] y = df.iloc[:, -1] X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42) return X_train, X_test, y_train, y_test當(dāng)談到數(shù)據(jù)科學(xué)領(lǐng)域時(shí)，大多數(shù)人都了解其中涉及的概念和術(shù)語(yǔ)，例如 X 和 Y。然而，對(duì)于初入這一領(lǐng)域的人來(lái)說(shuō)，是否將 CSV 文件的路徑命名為d是一個(gè)好的做法呢？另外，將特征命名為 X，將目標(biāo)命名為 y 是一個(gè)好的做法嗎？或許我們可以通過(guò)一個(gè)更具意義的例子來(lái)進(jìn)一步理解：
import pandas as pdfrom sklearn.model_selection import train_test_splitdef load_data_and_split_into_train_test(dataset_path): data_frame = pd.read_csv(dataset_path) features = data_frame.iloc[:, :-1] target = data_frame.iloc[:, -1] features_train, features_test, target_train, target_test = train_test_split(features, target, test_size=0.2, random_state=42) return features_train, features_test, target_train, target_test這樣就更容易理解了。即使沒(méi)有使用過(guò) pandas 和 train_test_split 的經(jīng)驗(yàn)，現(xiàn)在也能清楚地看到，這個(gè)函數(shù)是用來(lái)從 CSV 文件中加載數(shù)據(jù)（存儲(chǔ)在 dataset_path 中指定的路徑下），然后從數(shù)據(jù)框中提取特征和目標(biāo)，最后計(jì)算訓(xùn)練集和測(cè)試集的特征和目標(biāo)。
這些變化使代碼更易讀和易懂，尤其是對(duì)于那些可能不熟悉機(jī)器學(xué)習(xí)代碼規(guī)范的人來(lái)說(shuō)。在這些代碼中，特征大多以X表示，目標(biāo)以y表示。
但也不要過(guò)度夸大命名，因?yàn)檫@并不會(huì)提供任何額外的信息。
來(lái)看另一個(gè)示例代碼片段：
import pandas as pdfrom sklearn.model_selection import train_test_splitdef load_data_from_csv_and_split_into_training_and_testing_sets(dataset_path_csv): data_frame_from_csv = pd.read_csv(dataset_path_csv) features_columns_data_frame = data_frame_from_csv.iloc[:, :-1] target_column_data_frame = data_frame_from_csv.iloc[:, -1] features_columns_data_frame_for_training, features_columns_data_frame_for_testing, target_column_data_frame_for_training, target_column_data_frame_for_testing = train_test_split(features_columns_data_frame, target_column_data_frame, test_size=0.2, random_state=42) return features_columns_data_frame_for_training, features_columns_data_frame_for_testing, target_column_data_frame_for_training, target_column_data_frame_for_testing用戶(hù)提到的代碼讓人感覺(jué)信息過(guò)多，卻沒(méi)有提供任何額外的信息，反而會(huì)分散讀者的注意力。因此，建議在函數(shù)中添加有意義的名稱(chēng)，以取得描述性和簡(jiǎn)潔性之間的平衡。至于是否需要說(shuō)明函數(shù)是從 CSV 加載數(shù)據(jù)集路徑，這取決于代碼的上下文和實(shí)際需求。
函數(shù)的規(guī)模與功能應(yīng)該恰當(dāng)?shù)卦O(shè)計(jì)。它們應(yīng)該保持簡(jiǎn)潔，不超過(guò)20行，并將大塊內(nèi)容分離到新的函數(shù)中。更重要的是，函數(shù)應(yīng)該只負(fù)責(zé)一件事，而不是多個(gè)任務(wù)。如果需要執(zhí)行其他任務(wù)，就應(yīng)該將其放到另一個(gè)函數(shù)中。舉個(gè)例子
import pandas as pdfrom sklearn.model_selection import train_test_splitfrom sklearn.preprocessing import StandardScalerdef load_clean_feature_engineer_and_split(data_path): # Load data df = pd.read_csv(data_path) # Clean data df.dropna(inplace=True) df = df[df['Age'] > 0] # Feature engineering df['AgeGroup'] = pd.cut(df['Age'], bins=[0, 18, 65, 99], labels=['child', 'adult', 'senior']) df['IsAdult'] = df['Age'] > 18 # Data preprocessing scaler = StandardScaler() df[['Age', 'Fare']] = scaler.fit_transform(df[['Age', 'Fare']]) # Split data features = df.drop('Survived', axis=1) target = df['Survived'] features_train, features_test, target_train, target_test = train_test_split(features, target, test_size=0.2, random_state=42) return features_train, features_test, target_train, target_test你有沒(méi)有注意到違反了上述規(guī)則的行為？
雖然這個(gè)函數(shù)并不冗長(zhǎng)，但明顯違反了一個(gè)函數(shù)只負(fù)責(zé)一件事的規(guī)則。另外，注釋表明這些代碼塊可以放在一個(gè)單獨(dú)的函數(shù)中，因?yàn)楦静恍枰獑涡凶⑨專(zhuān)ㄏ乱还?jié)將詳細(xì)介紹）。
一個(gè)重構(gòu)后的示例：
import pandas as pdfrom sklearn.model_selection import train_test_splitfrom sklearn.preprocessing import StandardScalerdef load_data(data_path): return pd.read_csv(data_path)def clean_data(df): df.dropna(inplace=True) df = df[df['Age'] > 0] return dfdef feature_engineering(df): df['AgeGroup'] = pd.cut(df['Age'], bins=[0, 18, 65, 99], labels=['child', 'adult', 'senior']) df['IsAdult'] = df['Age'] > 18 return dfdef preprocess_features(df): scaler = StandardScaler() df[['Age', 'Fare']] = scaler.fit_transform(df[['Age', 'Fare']]) return dfdef split_data(df, target_name='Survived'): features = df.drop(target_name, axis=1) target = df[target_name] return train_test_split(features, target, test_size=0.2, random_state=42)if __name__ == "__main__": data_path = 'data.csv' df = load_data(data_path) df = clean_data(df) df = feature_engineering(df) df = preprocess_features(df) X_train, X_test, y_train, y_test = split_data(df)在這個(gè)經(jīng)過(guò)重構(gòu)的代碼片段中，每個(gè)函數(shù)只做一件事，這樣就更容易閱讀代碼了。測(cè)試本身也變得更容易了，因?yàn)槊總€(gè)函數(shù)都可以獨(dú)立于其他函數(shù)進(jìn)行測(cè)試。
甚至連注釋也不再需要了，因?yàn)楝F(xiàn)在函數(shù)名本身就像是注釋。
有時(shí)注釋是有用的，但有時(shí)它們只是糟糕代碼的標(biāo)志。
正確使用注釋是為了彌補(bǔ)我們無(wú)法用代碼表達(dá)的缺陷。
當(dāng)需要在代碼中添加注釋時(shí)，可以考慮是否真的需要它，或者是否可以將其放入一個(gè)新函數(shù)中，并為函數(shù)命名，這樣就能清楚地知道發(fā)生了什么，而注釋并不是必需的。
來(lái)修改一下之前函數(shù)一章中的代碼示例：
import pandas as pdfrom sklearn.model_selection import train_test_splitfrom sklearn.preprocessing import StandardScalerdef load_clean_feature_engineer_and_split(data_path): # Load data df = pd.read_csv(data_path) # Clean data df.dropna(inplace=True) df = df[df['Age'] > 0] # Feature engineering df['AgeGroup'] = pd.cut(df['Age'], bins=[0, 18, 65, 99], labels=['child', 'adult', 'senior']) df['IsAdult'] = df['Age'] > 18 # Data preprocessing scaler = StandardScaler() df[['Age', 'Fare']] = scaler.fit_transform(df[['Age', 'Fare']]) # Split data features = df.drop('Survived', axis=1) target = df['Survived'] features_train, features_test, target_train, target_test = train_test_split(features, target, test_size=0.2, random_state=42) return features_train, features_test, target_train, target_test代碼中注釋描述了每個(gè)代碼塊的作用，但實(shí)際上，注釋只是糟糕代碼的一個(gè)指標(biāo)。根據(jù)前一章的建議，將這些代碼塊放入單獨(dú)的函數(shù)中，并為每個(gè)函數(shù)起一個(gè)描述性的名稱(chēng)，這樣可以提高代碼的可讀性，減少對(duì)注釋的需求。
import pandas as pdfrom sklearn.model_selection import train_test_splitfrom sklearn.preprocessing import StandardScalerdef load_data(data_path): return pd.read_csv(data_path)def clean_data(df): df.dropna(inplace=True) df = df[df['Age'] > 0] return dfdef feature_engineering(df): df['AgeGroup'] = pd.cut(df['Age'], bins=[0, 18, 65, 99], labels=['child', 'adult', 'senior']) df['IsAdult'] = df['Age'] > 18 return dfdef preprocess_features(df): scaler = StandardScaler() df[['Age', 'Fare']] = scaler.fit_transform(df[['Age', 'Fare']]) return dfdef split_data(df, target_name='Survived'): features = df.drop('Survived', axis=1) target = df['Survived'] return train_test_split(features, target, test_size=0.2, random_state=42)if __name__ == "__main__": data_path = 'data.csv' df = load_data(data_path) df = clean_data(df) df = feature_engineering(df) df = preprocess_features(df) X_train, X_test, y_train, y_test = split_data(df)代碼現(xiàn)在看起來(lái)像一個(gè)連貫的故事，不需要注釋就可以清楚地了解發(fā)生了什么。但還缺少最后一部分：文檔字符串。文檔字符串是 Python 的標(biāo)準(zhǔn)，旨在提供可讀性和可理解性的代碼。每個(gè)生產(chǎn)代碼中的函數(shù)都應(yīng)該包含文檔字符串，描述其意圖、輸入?yún)?shù)和返回值信息。這些文檔字符串可以直接用于 Sphinx 等工具，其目的是為代碼創(chuàng)建文檔。
將文檔字符串添加到上述代碼片段中：
import pandas as pdfrom sklearn.model_selection import train_test_splitfrom sklearn.preprocessing import StandardScalerdef load_data(data_path): """ Load data from a CSV file into a pandas DataFrame. Args: data_path (str): The file path to the dataset. Returns: DataFrame: The loaded dataset. """ return pd.read_csv(data_path)def clean_data(df): """ Clean the DataFrame by removing rows with missing values and filtering out non-positive ages. Args: df (DataFrame): The input dataset. Returns: DataFrame: The cleaned dataset. """ df.dropna(inplace=True) df = df[df['Age'] > 0] return dfdef feature_engineering(df): """ Perform feature engineering on the DataFrame, including age grouping and adult identification. Args: df (DataFrame): The input dataset. Returns: DataFrame: The dataset with new features added. """ df['AgeGroup'] = pd.cut(df['Age'], bins=[0, 18, 65, 99], labels=['child', 'adult', 'senior']) df['IsAdult'] = df['Age'] > 18 return dfdef preprocess_features(df): """ Preprocess features by standardizing the 'Age' and 'Fare' columns using StandardScaler. Args: df (DataFrame): The input dataset. Returns: DataFrame: The dataset with standardized features. """ scaler = StandardScaler() df[['Age', 'Fare']] = scaler.fit_transform(df[['Age', 'Fare']]) return dfdef split_data(df, target_name='Survived'): """ Split the dataset into training and testing sets. Args: df (DataFrame): The input dataset. target_name (str): The name of the target variable column. Returns: tuple: Contains the training features, testing features, training target, and testing target datasets. """ features = df.drop(target_name, axis=1) target = df[target_name] return train_test_split(features, target, test_size=0.2, random_state=42)if __name__ == "__main__": data_path = 'data.csv' df = load_data(data_path) df = clean_data(df) df = feature_engineering(df) df = preprocess_features(df) X_train, X_test, y_train, y_test = split_data(df)集成開(kāi)發(fā)環(huán)境（如 VSCode）通常會(huì)提供 docstrings 擴(kuò)展功能，以便在函數(shù)定義下方添加多行字符串時(shí)自動(dòng)添加 docstrings。
這可以幫助你快速獲得所選的正確格式。
格式化是一個(gè)非常關(guān)鍵的概念。
代碼的閱讀頻率比編寫(xiě)頻率高。避免人們閱讀不規(guī)范和難以理解的代碼。
在 Python 中有一個(gè) PEP 8 樣式指南[1]，可用于改善代碼的可讀性。
樣式指南包括如下重要規(guī)則：
但請(qǐng)記住，格式化規(guī)則旨在提高代碼可讀性。有時(shí)，嚴(yán)格遵循規(guī)則可能不合理，會(huì)降低代碼的可讀性。此時(shí)可以忽略某些規(guī)則。
《清潔代碼》一書(shū)中提到的其他重要格式化規(guī)則包括：
因此，與團(tuán)隊(duì)一起決定遵守的規(guī)則，并堅(jiān)持執(zhí)行！您可以利用集成開(kāi)發(fā)環(huán)境的擴(kuò)展功能來(lái)支持準(zhǔn)則遵守。例如，VSCode 提供了多種擴(kuò)展。您可以使用 Pylint[2] 和 autopep8[3] 等 Python 軟件包來(lái)格式化您的 Python 腳本。Pylint 是一個(gè)靜態(tài)代碼分析器，自動(dòng)對(duì)代碼進(jìn)行評(píng)分，而autopep8可以自動(dòng)格式化代碼，使其符合PEP8標(biāo)準(zhǔn)。
使用前面的代碼片段來(lái)進(jìn)一步了解。
import pandas as pdfrom sklearn.model_selection import train_test_splitfrom sklearn.preprocessing import StandardScalerdef load_data(data_path): return pd.read_csv(data_path)def clean_data(df): df.dropna(inplace=True) df = df[df['Age'] > 0] return dfdef feature_engineering(df): df['AgeGroup'] = pd.cut(df['Age'], bins=[0, 18, 65, 99], labels=['child', 'adult', 'senior']) df['IsAdult'] = df['Age'] > 18 return dfdef preprocess_features(df): scaler = StandardScaler() df[['Age', 'Fare']] = scaler.fit_transform(df[['Age', 'Fare']]) return dfdef split_data(df, target_name='Survived'): features = df.drop(target_name, axis=1) target = df[target_name] return train_test_split(features, target, test_size=0.2, random_state=42)if __name__ == "__main__": data_path = 'data.csv' df = load_data(data_path) df = clean_data(df) df = feature_engineering(df) df = preprocess_features(df) X_train, X_test, y_train, y_test = split_data(df)將其保存到名為 train.py 的文件中，并運(yùn)行 Pylint 來(lái)檢查該代碼段的得分：
pylint train.py輸出結(jié)果
************* Module traintrain.py:29:0: W0311: Bad indentation. Found 2 spaces, expected 4 (bad-indentation)train.py:30:0: W0311: Bad indentation. Found 2 spaces, expected 4 (bad-indentation)train.py:31:0: W0311: Bad indentation. Found 2 spaces, expected 4 (bad-indentation)train.py:32:0: W0311: Bad indentation. Found 2 spaces, expected 4 (bad-indentation)train.py:33:0: W0311: Bad indentation. Found 2 spaces, expected 4 (bad-indentation)train.py:34:0: C0304: Final newline missing (missing-final-newline)train.py:34:0: W0311: Bad indentation. Found 2 spaces, expected 4 (bad-indentation)train.py:1:0: C0114: Missing module docstring (missing-module-docstring)train.py:5:0: C0116: Missing function or method docstring (missing-function-docstring)train.py:5:14: W0621: Redefining name 'data_path' from outer scope (line 29) (redefined-outer-name)train.py:8:0: C0116: Missing function or method docstring (missing-function-docstring)train.py:8:15: W0621: Redefining name 'df' from outer scope (line 30) (redefined-outer-name)train.py:13:0: C0116: Missing function or method docstring (missing-function-docstring)train.py:13:24: W0621: Redefining name 'df' from outer scope (line 30) (redefined-outer-name)train.py:18:0: C0116: Missing function or method docstring (missing-function-docstring)train.py:18:24: W0621: Redefining name 'df' from outer scope (line 30) (redefined-outer-name)train.py:23:0: C0116: Missing function or method docstring (missing-function-docstring)train.py:23:15: W0621: Redefining name 'df' from outer scope (line 30) (redefined-outer-name)train.py:29:2: C0103: Constant name "data_path" doesn't conform to UPPER_CASE naming style (invalid-name)------------------------------------------------------------------Your code has been rated at 3.21/10滿(mǎn)分 10 分，只有 3.21 分。
你可以選擇手動(dòng)修復(fù)這些問(wèn)題然后重新運(yùn)行，或者使用autopep8軟件包來(lái)自動(dòng)解決一些問(wèn)題。下面我們選擇第二種方法。
autopep8 --in-place --aggressive --aggressive train.py現(xiàn)在的 train.py 腳本如下所示：
import pandas as pdfrom sklearn.model_selection import train_test_splitfrom sklearn.preprocessing import StandardScalerdef load_data(data_path): return pd.read_csv(data_path)def clean_data(df): df.dropna(inplace=True) df = df[df['Age'] > 0] return dfdef feature_engineering(df): df['AgeGroup'] = pd.cut( df['Age'], bins=[ 0, 18, 65, 99], labels=[ 'child', 'adult', 'senior']) df['IsAdult'] = df['Age'] > 18 return dfdef preprocess_features(df): scaler = StandardScaler() df[['Age', 'Fare']] = scaler.fit_transform(df[['Age', 'Fare']]) return dfdef split_data(df, target_name='Survived'): features = df.drop(target_name, axis=1) target = df[target_name] return train_test_split(features, target, test_size=0.2, random_state=42)if __name__ == "__main__": data_path = 'data.csv' df = load_data(data_path) df = clean_data(df) df = feature_engineering(df) df = preprocess_features(df) X_train, X_test, y_train, y_test = split_data(df)再次運(yùn)行 Pylint 后，我們得到了 5.71 分（滿(mǎn)分 10 分），這主要是由于缺少函數(shù)的文檔說(shuō)明：
************* Module traintrain.py:1:0: C0114: Missing module docstring (missing-module-docstring)train.py:6:0: C0116: Missing function or method docstring (missing-function-docstring)train.py:6:14: W0621: Redefining name 'data_path' from outer scope (line 38) (redefined-outer-name)train.py:10:0: C0116: Missing function or method docstring (missing-function-docstring)train.py:10:15: W0621: Redefining name 'df' from outer scope (line 39) (redefined-outer-name)train.py:16:0: C0116: Missing function or method docstring (missing-function-docstring)train.py:16:24: W0621: Redefining name 'df' from outer scope (line 39) (redefined-outer-name)train.py:25:0: C0116: Missing function or method docstring (missing-function-docstring)train.py:25:24: W0621: Redefining name 'df' from outer scope (line 39) (redefined-outer-name)train.py:31:0: C0116: Missing function or method docstring (missing-function-docstring)train.py:31:15: W0621: Redefining name 'df' from outer scope (line 39) (redefined-outer-name)train.py:38:4: C0103: Constant name "data_path" doesn't conform to UPPER_CASE naming style (invalid-name)------------------------------------------------------------------Your code has been rated at 5.71/10 (previous run: 3.21/10, +2.50)現(xiàn)在我已經(jīng)添加了文檔說(shuō)明，并修復(fù)了最后的缺失點(diǎn)。
現(xiàn)在的最終代碼是這樣的：
"""This script aims at providing an end-to-end training pipeline.Author: Patrick Date: 2/14/2024"""import pandas as pdfrom sklearn.model_selection import train_test_splitfrom sklearn.preprocessing import StandardScalerdef load_data(data_path): """ Load dataset from a specified CSV file into a pandas DataFrame. Args: data_path (str): The file path to the dataset. Returns: DataFrame: The loaded dataset. """ return pd.read_csv(data_path)def clean_data(df): """ Clean the input DataFrame by removing rows with missing values and filtering out entries with non-positive ages. Args: df (DataFrame): The input dataset. Returns: DataFrame: The cleaned dataset. """ df.dropna(inplace=True) df = df[df['Age'] > 0] return dfdef feature_engineering(df): """ Perform feature engineering on the DataFrame, including creating age groups and determining if the individual is an adult. Args: df (DataFrame): The input dataset. Returns: DataFrame: The dataset with new features added. """ df['AgeGroup'] = pd.cut( df['Age'], bins=[ 0, 18, 65, 99], labels=[ 'child', 'adult', 'senior']) df['IsAdult'] = df['Age'] > 18 return dfdef preprocess_features(df): """ Preprocess the 'Age' and 'Fare' features of the DataFrame using StandardScaler to standardize the features. Args: df (DataFrame): The input dataset. Returns: DataFrame: The dataset with standardized features. """ scaler = StandardScaler() df[['Age', 'Fare']] = scaler.fit_transform(df[['Age', 'Fare']]) return dfdef split_data(df, target_name='Survived'): """ Split the DataFrame into training and testing sets. Args: df (DataFrame): The dataset to split. target_name (str, optional): The name of the target variable column. Defaults to 'Survived'. Returns: tuple: The training and testing features and target datasets. """ features = df.drop(target_name, axis=1) target = df[target_name] return train_test_split(features, target, test_size=0.2, random_state=42)if __name__ == "__main__": data = load_data("data.csv") data = clean_data(data) data = feature_engineering(data) data = preprocess_features(data) X_train, X_test, y_train, y_test = split_data(data)運(yùn)行 Pylint 現(xiàn)在返回 10 分：
pylint train.py-------------------------------------------------------------------Your code has been rated at 10.00/10 (previous run: 7.50/10, +2.50)這突出顯示了 Pylint 的功能之強(qiáng)大，它可以幫助您簡(jiǎn)化代碼并快速符合 PEP8 標(biāo)準(zhǔn)。
錯(cuò)誤處理是另一個(gè)關(guān)鍵概念。它能確保你的代碼在遇到意外情況時(shí)不會(huì)崩潰或產(chǎn)生錯(cuò)誤結(jié)果。
舉個(gè)例子，假設(shè)您在API后端部署了一個(gè)模型，用戶(hù)可以向該部署的模型發(fā)送數(shù)據(jù)。然而，用戶(hù)可能會(huì)發(fā)送錯(cuò)誤的數(shù)據(jù)，而你的應(yīng)用程序如果崩潰了，可能會(huì)給用戶(hù)留下不好的印象，并可能因此責(zé)備您的應(yīng)用程序開(kāi)發(fā)不到位。
如果用戶(hù)能夠獲取明確的錯(cuò)誤代碼和相關(guān)信息，清晰地指出他們的錯(cuò)誤，那就更好了。這正是Python中異常的作用所在。
舉例來(lái)說(shuō)，用戶(hù)可以上傳一個(gè)CSV文件到您的應(yīng)用程序，將其加載到pandas數(shù)據(jù)框架中，然后將數(shù)據(jù)傳給模型進(jìn)行預(yù)測(cè)。這樣，您可以擁有類(lèi)似下面這樣的函數(shù)：
import pandas as pddef load_data(data_path): """ Load dataset from a specified CSV file into a pandas DataFrame. Args: data_path (str): The file path to the dataset. Returns: DataFrame: The loaded dataset. """ return pd.read_csv(data_path)到目前為止，一切順利。但如果用戶(hù)沒(méi)有提供 CSV 文件，會(huì)發(fā)生什么情況呢？
你的程序?qū)⒈罎ⅲ⒊霈F(xiàn)以下錯(cuò)誤信息：
FileNotFoundError: [Errno 2] No such file or directory: 'data.csv'你正在使用API，它只會(huì)以HTTP 500代碼響應(yīng)用戶(hù)，告訴他們"服務(wù)器內(nèi)部出錯(cuò)"。用戶(hù)可能會(huì)因此責(zé)怪您的應(yīng)用程序，因?yàn)樗麄儫o(wú)法確定自己是否對(duì)該錯(cuò)誤負(fù)有責(zé)任。更好的處理方法是添加一個(gè)try-except塊，并捕獲FileNotFoundError以正確處理這種情況。
import pandas as pdimport loggingdef load_data(data_path): """ Load dataset from a specified CSV file into a pandas DataFrame. Args: data_path (str): The file path to the dataset. Returns: DataFrame: The loaded dataset. """ try: return pd.read_csv(data_path) except FileNotFoundError: logging.error("The file at path %s does not exist. Please ensure that you have uploaded the file properly.", data_path)我們目前只能記錄該錯(cuò)誤消息。最佳做法是定義一個(gè)自定義異常，然后在應(yīng)用程序接口中進(jìn)行處理，以向用戶(hù)返回特定的錯(cuò)誤代碼。
import pandas as pdimport loggingclass DataLoadError(Exception): """Exception raised when the data cannot be loaded.""" def __init__(self, message="Data could not be loaded"): self.message = message super().__init__(self.message)def load_data(data_path): """ Load dataset from a specified CSV file into a pandas DataFrame. Args: data_path (str): The file path to the dataset. Returns: DataFrame: The loaded dataset. """ try: return pd.read_csv(data_path) except FileNotFoundError: logging.error("The file at path %s does not exist. Please ensure that you have uploaded the file properly.", data_path) raise DataLoadError(f"The file at path {data_path} does not exist. Please ensure that you have uploaded the file properly.")然后，在應(yīng)用程序接口的主要函數(shù)中：
try: df = load_data('path/to/data.csv') # Further processing and model predictionexcept DataLoadError as e: # Return a response to the user with the error message # For example: return Response({"error": str(e)}, status=400)用戶(hù)將收到 400 錯(cuò)誤代碼（錯(cuò)誤請(qǐng)求），并將收到有關(guān)錯(cuò)誤原因的錯(cuò)誤消息。
現(xiàn)在他了解了應(yīng)該怎么做，并不會(huì)再責(zé)備程序工作不正常。
面向?qū)ο缶幊?OOP)是Python中一個(gè)重要的編程范式，即使是初學(xué)者也應(yīng)該熟悉。那么，什么是OOP呢?
面向?qū)ο缶幊淌且环N將數(shù)據(jù)和行為封裝到單個(gè)對(duì)象中的編程方式，為程序提供了清晰的結(jié)構(gòu)。
采用OOP有以下幾個(gè)主要好處:
OOP還有其他一些優(yōu)點(diǎn)，上述幾點(diǎn)是最為關(guān)鍵的。
現(xiàn)在讓我們看一個(gè)簡(jiǎn)單的例子，我們創(chuàng)建了一個(gè)名為T(mén)rainingPipeline的類(lèi)，包含幾個(gè)基本函數(shù):
from abc import ABC, abstractmethodclass TrainingPipeline(ABC): def __init__(self, data_path, target_name): """ Initialize the TrainingPipeline. Args: data_path (str): The file path to the dataset. target_name (str): Name of the target column. """ self.data_path = data_path self.target_name = target_name self.data = None self.X_train = None self.X_test = None self.y_train = None self.y_test = None @abstractmethod def load_data(self): """Load dataset from data path.""" pass @abstractmethod def clean_data(self): """Clean the data.""" pass @abstractmethod def feature_engineering(self): """Perform feature engineering.""" pass @abstractmethod def preprocess_features(self): """Preprocess features.""" pass @abstractmethod def split_data(self): """Split data into training and testing sets.""" pass def run(self): """Run the training pipeline.""" self.load_data() self.clean_data() self.feature_engineering() self.preprocess_features() self.split_data()這是一個(gè)抽象基類(lèi)，只定義了從基類(lèi)派生出來(lái)的類(lèi)必須實(shí)現(xiàn)的抽象方法。
這對(duì)于定義所有子類(lèi)都必須遵循的藍(lán)圖或模板非常有用。
下面是一個(gè)子類(lèi)示例：
import pandas as pdfrom sklearn.preprocessing import StandardScalerclass ChurnPredictionTrainPipeline(TrainingPipeline): def load_data(self): """Load dataset from data path.""" self.data = pd.read_csv(self.data_path) def clean_data(self): """Clean the data.""" self.data.dropna(inplace=True) def feature_engineering(self): """Perform feature engineering.""" categorical_cols = self.data.select_dtypes(include=['object', 'category']).columns self.data = pd.get_dummies(self.data, columns=categorical_cols, drop_first=True) def preprocess_features(self): """Preprocess features.""" numerical_cols = self.data.select_dtypes(include=['int64', 'float64']).columns scaler = StandardScaler() self.data[numerical_cols] = scaler.fit_transform(self.data[numerical_cols]) def split_data(self): """Split data into training and testing sets.""" features = self.data.drop(self.target_name, axis=1) target = self.data[self.target_name] self.features_train, self.features_test, self.target_train, self.target_test = train_test_split(features, target, test_size=0.2, random_state=42)這樣做的好處是，你可以創(chuàng)建一個(gè)自動(dòng)調(diào)用訓(xùn)練管道方法的應(yīng)用程序，還可以創(chuàng)建不同的訓(xùn)練管道類(lèi)。它們始終是兼容的，并且必須遵循抽象基類(lèi)中定義的藍(lán)圖。
測(cè)試可以決定整個(gè)項(xiàng)目的成敗。
測(cè)試確實(shí)可能會(huì)增加一些開(kāi)發(fā)時(shí)間投入，但從長(zhǎng)遠(yuǎn)來(lái)看，它能夠極大地提高代碼質(zhì)量、可維護(hù)性和可靠性。
測(cè)試對(duì)于確保項(xiàng)目的成功至關(guān)重要，盡管一開(kāi)始編寫(xiě)測(cè)試代碼會(huì)耗費(fèi)一些時(shí)間，但這是一種非常值得的投資。不編寫(xiě)測(cè)試可能會(huì)在短期內(nèi)加快開(kāi)發(fā)速度，但從長(zhǎng)遠(yuǎn)來(lái)看，缺乏測(cè)試會(huì)帶來(lái)嚴(yán)重的代價(jià):
因此，遵循 TDD 原則對(duì)于提高代碼質(zhì)量和開(kāi)發(fā)效率至關(guān)重要。TDD 的三個(gè)核心原則是:
這種測(cè)試先行的模式能促使開(kāi)發(fā)者先思考代碼設(shè)計(jì)。
Python 擁有諸如 unittest 和 pytest 等優(yōu)秀測(cè)試框架，其中 pytest 因其簡(jiǎn)潔語(yǔ)法而更加易用。盡管短期增加了開(kāi)發(fā)量，但測(cè)試絕對(duì)是保證項(xiàng)目長(zhǎng)期成功所必需的。
再次看看前一章中的 ChurnPredictionTrainPipeline 類(lèi)：
import pandas as pdfrom sklearn.preprocessing import StandardScalerclass ChurnPredictionTrainPipeline(TrainingPipeline): def load_data(self): """Load dataset from data path.""" self.data = pd.read_csv(self.data_path) ...使用 pytest 為加載數(shù)據(jù)添加單元測(cè)試：
import osimport shutilimport loggingfrom unittest.mock import patchimport joblibimport pytestimport numpy as npimport pandas as pdfrom sklearn.datasets import make_classificationfrom sklearn.model_selection import train_test_splitfrom sklearn.ensemble import RandomForestClassifierfrom sklearn.linear_model import LogisticRegressionfrom churn_library import ChurnPredictor@pytest.fixturedef path(): """ Return the path to the test csv data file. """ return r"./data/bank_data.csv"def test_import_data_returns_dataframe(path): """ Test that import data can load the CSV file into a pandas dataframe. """ churn_predictor = ChurnPredictionTrainPipeline(path, "Churn") churn_predictor.load_data() assert isinstance(churn_predictor.data, pd.DataFrame)def test_import_data_raises_exception(): """ Test that exception of "FileNotFoundError" gets raised in case the CSV file does not exist. """ with pytest.raises(FileNotFoundError): churn_predictor = ChurnPredictionTrainPipeline("non_existent_file.csv", "Churn") churn_predictor.load_data()def test_import_data_reads_csv(path): """ Test that the pandas.read_csv function gets called. """ with patch("pandas.read_csv") as mock_csv: churn_predictor = ChurnPredictionTrainPipeline(path, "Churn") churn_predictor.load_data() mock_csv.assert_called_once_with(path)這些單元測(cè)試包括
這個(gè)過(guò)程并不完全是 TDD，因?yàn)樵谔砑訂卧獪y(cè)試之前，我已經(jīng)開(kāi)發(fā)了代碼。但在理想情況下，你甚至可以在實(shí)現(xiàn) load_data 函數(shù)之前編寫(xiě)這些單元測(cè)試。
四條簡(jiǎn)單設(shè)計(jì)規(guī)則,目的是讓代碼更加簡(jiǎn)潔、可讀和易維護(hù)。這四條規(guī)則是:
前三條規(guī)則側(cè)重于代碼重構(gòu)方面。在最初編碼時(shí)不要過(guò)于追求完美，可以先寫(xiě)出簡(jiǎn)單甚至"丑陋"的代碼，待代碼能夠運(yùn)行后，再通過(guò)重構(gòu)來(lái)遵循上述規(guī)則，使代碼變得優(yōu)雅。
推薦"先實(shí)現(xiàn)，后重構(gòu)"的編程方式。不要一開(kāi)始就過(guò)分追求完美，而是先讓代碼運(yùn)行起來(lái)，功能被實(shí)現(xiàn)，之后再反復(fù)重構(gòu)，循序漸進(jìn)地遵從這四條簡(jiǎn)單設(shè)計(jì)原則，從而提高代碼質(zhì)量。
編寫(xiě)簡(jiǎn)潔代碼對(duì)軟件項(xiàng)目的成功至關(guān)重要，但這需要嚴(yán)謹(jǐn)?shù)膽B(tài)度和持續(xù)的練習(xí)。作為數(shù)據(jù)科學(xué)家，我們往往更關(guān)注在Jupyter Notebooks中運(yùn)行代碼、尋找好的模型和獲取理想指標(biāo)，而忽視了代碼的整潔度。但是，編寫(xiě)簡(jiǎn)潔代碼也是數(shù)據(jù)科學(xué)家的必修課，因?yàn)檫@能確保模型更快地投入生產(chǎn)環(huán)境。
當(dāng)編寫(xiě)需要重復(fù)使用的代碼時(shí)，我們應(yīng)當(dāng)堅(jiān)持編寫(xiě)簡(jiǎn)潔代碼。起步可以從簡(jiǎn)單開(kāi)始，不要一開(kāi)始就過(guò)于追求完美，而是要反復(fù)打磨代碼。永遠(yuǎn)不要忘記為函數(shù)編寫(xiě)單元測(cè)試，以確保功能的正常運(yùn)行，避免將來(lái)擴(kuò)展時(shí)出現(xiàn)重大問(wèn)題。
堅(jiān)持一些原則，比如消除重復(fù)代碼、體現(xiàn)代碼意圖等，能讓你遠(yuǎn)離"永遠(yuǎn)不要改變正在運(yùn)行的系統(tǒng)"的思維定式。這些原則我正在學(xué)習(xí)并應(yīng)用到日常工作中，它們確實(shí)很有幫助，但全面掌握需要漫長(zhǎng)的過(guò)程和持續(xù)的努力。
最后，要盡可能自動(dòng)化，利用集成開(kāi)發(fā)環(huán)境提供的擴(kuò)展功能，來(lái)幫助遵守清潔代碼規(guī)則，提高工作效率。
[1]PEP 8 樣式指南: https://peps.python.org/pep-0008/
[2]Pylint: https://pylint.readthedocs.io/en/stable/
[3]autopep8: https://pypi.org/project/autopep8/
本文鏈接：http://m.www897cc.com/showinfo-26-92161-0.html我發(fā)現(xiàn)了用 Python 編寫(xiě)簡(jiǎn)潔代碼的秘訣！
聲明：本網(wǎng)頁(yè)內(nèi)容旨在傳播知識(shí)，若有侵權(quán)等問(wèn)題請(qǐng)及時(shí)與本網(wǎng)聯(lián)系，我們將在第一時(shí)間刪除處理。郵件：2376512515@qq.com
Copyright ? 2016-2023 天津谷騏科技有限公司 版權(quán)所有 sitemap.xml
違法及侵權(quán)請(qǐng)聯(lián)系：2376512515@qq.com 津ICP備18001702號(hào)
津公網(wǎng)安備 12010102000574號(hào)