与谷歌Colab一起使用Git的方法

Pra*_*nth 43 google-colaboratory

有没有推荐的方法将git与colab集成?

例如,是否可以处理谷歌源存储库或类似的代码?

谷歌驱动器和云存储都不能用于git功能.

所以我想知道是否有办法继续这样做?

Bob*_*ith 28

git安装在计算机上,您可以使用它!来调用shell命令.

例如,要克隆git存储库:

!git clone https://github.com/fastai/courses.git
Run Code Online (Sandbox Code Playgroud)

这是一个完整的示例,它克隆存储库并加载存储在其中的Excel文件. https://colab.research.google.com/notebook#fileId=1v-yZk-W4YXOxLTLi7bekDw2ZWZXWW216

  • 有没有办法在克隆私人回购时输入用户名和密码? (8认同)
  • 如何将结果从Colab返回到github。我主要是为了检查点文件来检索模型以在本地计算机上进行推理 (3认同)
  • 在 github 设置的 Dev 选项卡中创建一个令牌并使用:`! git 克隆 https://TOKEN@github.com/username/repository.git`。 (3认同)

小智 22

如果要克隆私有存储库,最快的方法是:

!git clone https://username:password@github.com/username/repository.git

  • 如果您的密码中有“@”,请将其替换为“%40”。 (5认同)
  • 我能找到的最安全的方法是使用上面定义的令牌并将其作为变量,这样它的痕迹在使用后就消失了。`从 IPython.display 导入clear_output;u = 输入();键=输入();clear_output() ` 运行 `!git clone https://$u:$key@github.com/rohitdavas/reponame.git` (5认同)
  • `u = '用户0'; p = 'pass0'; !git clone https://$u:$p@github.com/$u/repository.git` 这应该可以工作,`{p} {u}`也可以。(!...不过应该另起一行。) (3认同)
  • 这几乎是重复另一个答案。但欢迎。 (2认同)
  • 有没有办法动态传递用户名和密码。使用变量? (2认同)

Far*_*han 12

\n

2021 年 9 月更新 \xe2\x80\x94 出于安全原因,现在 github 使用时已弃用密码。请使用Personal Access Token\xe2\x80\x94 转至 github.com -> 设置 -> 开发人员设置 -> 个人访问令牌并生成用于所需目的的令牌。使用此密码代替您的密码来完成本教程中提到的所有任务!

\n
\n

有关更多详细信息,您还可以参阅我在 Medium 上的文章:https://medium.com/geekculture/using-git-github-on-google-colaboratory-7ef3b76fe61b

\n

没有一个答案提供像这样直接的答案:

\n

GitColab

\n

也许这就是您正在寻找的答案..

\n

适用于公共和私人存储库的 colab,并且不更改/跳过任何步骤:(替换全部{vars}

\n

TL;DR 完整流程:

\n
!git clone https://{your_username}:{your_password}@github.com/{destination_repo_username}/{destination_repo_projectname}.git\n%cd /content/{destination_repo_username}\n\n!git config --global user.name "{your_username}"\n!git config --global user.email "{your_email_id}"\n!git config --global user.password "{your_password}"\n
Run Code Online (Sandbox Code Playgroud)\n

进行更改然后运行:

\n
!git add .\n!git commit -m "{Message}"\n!git push\n
Run Code Online (Sandbox Code Playgroud)\n

克隆存储库:

\n
!git clone https://{your_username}:{your_password}@github.com/{destination_repo_username}/{destination_repo_projectname}.git\n
Run Code Online (Sandbox Code Playgroud)\n

将目录更改为

\n

%cd使用jupyter 笔记本的line magic 命令将目录更改为 {destination_repo_username} 。

\n
%cd /content/{destination_repo_username}\n
Run Code Online (Sandbox Code Playgroud)\n

核实!

\n

\n

健全性检查,看看一切是否完美!

\n
!git pull\n
Run Code Online (Sandbox Code Playgroud)\n

如果克隆后未对远程 git 存储库进行任何更改,则显示的输出应如下:

\n
Already up to date.\n
Run Code Online (Sandbox Code Playgroud)\n

地位

\n

同样检查暂存/未暂存更改的状态。

\n
!git status\n
Run Code Online (Sandbox Code Playgroud)\n

它应该显示此内容,并选择默认分支:

\n
On branch main\nYour branch is up to date with \'origin/main\'.\n\nnothing to commit, working tree clean\n
Run Code Online (Sandbox Code Playgroud)\n

检查较旧的日志

\n

检查您之前在存储库上所做的提交:

\n
!git log -n 4\n
Run Code Online (Sandbox Code Playgroud)\n

输出 Git 提交 ID 和日志:

\n
commit 18ccf27c8b2d92b560e6eeab2629ba0c6ea422a5 (HEAD -> main, origin/main, origin/HEAD)\nAuthor: Farhan Hai Khan <njrfarhandasilva10@gmail.com>\nDate:   Mon May 31 00:12:14 2021 +0530\n\n    Create README.md\n\ncommit bd6ee6d4347eca0e3676e88824c8e1118cfbff6b\nAuthor: khanfarhan10 <njrfarhandasilva10@gmail.com>\nDate:   Sun May 30 18:40:16 2021 +0000\n\n    Add Zip COVID\n\ncommit 8a3a12863a866c9d388cbc041a26d49aedfa4245\nAuthor: khanfarhan10 <njrfarhandasilva10@gmail.com>\nDate:   Sun May 30 18:03:46 2021 +0000\n\n    Add COVID Data\n\ncommit 6a16dc7584ba0d800eede70a217d534a24614cad\nAuthor: khanfarhan10 <njrfarhandasilva10@gmail.com>\nDate:   Sun May 30 16:04:20 2021 +0000\n\n    Removed sample_data using colab (testing)\n
Run Code Online (Sandbox Code Playgroud)\n

在本地存储库中进行更改

\n

从本地存储库目录进行更改。

\n

这些可能包括编辑、删除、编辑。

\n
专业提示:如果您愿意,可以通过以下方式将内容从驱动器复制粘贴到 git 存储库:
\n

挂载 Google 云端硬盘:

\n
from google.colab import drive\ndrive.mount(\'/content/gdrive\')\n
Run Code Online (Sandbox Code Playgroud)\n

使用shutil复制内容:

\n
import shutil\n\n# For a folder:\nshutil.copytree(src_folder,des_folder)\n\n# For a file:\nshutil.copy(src_file,des_file)\n\n# Create a ZipFile\nshutil.make_archive(archive_name, \'zip\', directory_to_zip)\n
Run Code Online (Sandbox Code Playgroud)\n

设置 Git 凭证

\n

告诉 Git 你是谁?

\n
!git config --global user.name "{your_username}"\n!git config --global user.email "{your_email_id}"\n!git config --global user.password "{your_password}"\n
Run Code Online (Sandbox Code Playgroud)\n

再次检查远程

\n

检查远程 url 是否设置和配置正确:

\n
!git remote -v\n
Run Code Online (Sandbox Code Playgroud)\n

如果配置正确,它应该输出以下内容:

\n
origin  https://{your_username}:{your_password}@github.com/{destination_repo_username}/{destination_repo_projectname}.git (fetch)\norigin  https://{your_username}:{your_password}@github.com/{destination_repo_username}/{destination_repo_projectname}.git (push)\n
Run Code Online (Sandbox Code Playgroud)\n

添加、提交、推送

\n

你知道该做什么。

\n
!git add .\n!git commit -m "{Message}"\n!git push\n
Run Code Online (Sandbox Code Playgroud)\n

享受!

\n


小智 10

为了保护您的帐户用户名和密码,您可以getPass在shell命令中使用并连接它们:

from getpass import getpass
import os

user = getpass('BitBucket user')
password = getpass('BitBucket password')
os.environ['BITBUCKET_AUTH'] = user + ':' + password

!git clone https://$BITBUCKET_AUTH@bitbucket.org/{user}/repository.git
Run Code Online (Sandbox Code Playgroud)


mou*_*afa 9

将私有存储库克隆到 google colab :

生成令牌:

Settings -> Developer settings -> Personal access tokens -> Generate new token
Run Code Online (Sandbox Code Playgroud)

复制令牌并克隆存储库(相应地替换用户名令牌

!git clone https://username:token@github.com/username/repo_name.git
Run Code Online (Sandbox Code Playgroud)


con*_* yu 7

使用git同步colab与github或gitlab的三个步骤。

  1. 生成私钥-公钥对。将私钥复制到系统 clibboard 以用于步骤 2。将公钥粘贴到 github 或 gitlab 中。

    在 Linux 中,ssh-keygen 可用于在 ~/.ssh 中生成密钥对。生成的私钥在文件 id_rsa 中,公钥在文件 id_rsa.pub 中。

  2. 在 Colab 中,执行

    key = \
    '''
    paste the private key here 
    (your id_rsa or id_ecdsa file in the .ssh directory, e.g.
    -----BEGIN EC PRIVATE KEY-----
    M..............................................................9
    ...............................................................J
    ..................................==
    -----END EC PRIVATE KEY-----
    '''
    ! mkdir -p /root/.ssh
    with open(r'/root/.ssh/id_rsa', 'w', encoding='utf8') as fh:
        fh.write(key)
    ! chmod 600 /root/.ssh/id_rsa
    ! ssh-keyscan github.com >> /root/.ssh/known_hosts 
    # test setup
    ! ssh -T git@github.com
    # if you see something like "Hi ffreemt! You've successfully 
    # authenticated, but GitHub does not provide shell access."
    # you are all set. You can tweak .ssh/config for multiple github accounts
    
    Run Code Online (Sandbox Code Playgroud)
  3. 像往常一样使用 git 拉/推。

相同的想法可用于 rsync(或 ssh)colab 和 HostA 之间,只需稍作更改:

  1. 生成私钥-公钥对。将私钥复制到系统clibboard 以供步骤2 使用。将公钥粘贴到HostA 中.ssh 中的authorized_keys。

  2. 在 Colab 中,执行

    key = \
    '''
    paste the private key here
    '''
    ! mkdir -p /root/.ssh
    with open(r'/root/.ssh/id_rsa', 'w', encoding='utf8') as fh:
        fh.write(key)
    ! chmod 600 /root/.ssh/id_rsa
    ! ssh -oStrictHostKeyChecking=no root@HostA hostnam  # ssh-keyscan 
    
    Run Code Online (Sandbox Code Playgroud)

HostA >> /root/.ssh/known_hosts 似乎不适用于 IP。

  1. 像往常一样使用 rsync 同步 colab 和 HostA 之间的文件。


Fad*_*ura 6

您可以使用ssh协议将专用存储库与colab连接

  1. 在您的本地计算机上生成ssh密钥对,不要忘了将
    短语保留为空,请查看本教程

  2. 上载到colab,查看以下截图

    from google.colab import files
    uploaded = files.upload()

  3. 将ssh kay对移动到/ root并连接到git

    • 删除以前的ssh文件
      ! rm -rf /root/.ssh/*
      ! mkdir /root/.ssh
    • 解压缩ssh文件
      ! tar -xvzf ssh.tar.gz
    • 复制到根目录
      ! cp ssh/* /root/.ssh && rm -rf ssh && rm -rf ssh.tar.gz ! chmod 700 /root/.ssh
    • 添加您的git服务器,例如gitlab作为ssh已知主机
      ! ssh-keyscan gitlab.com >> /root/.ssh/known_hosts
      ! chmod 644 /root/.ssh/known_hosts
    • 设置你的git帐户
      ! git config --global user.email "email"
      ! git config --global user.name "username"
    • 最终连接到您的git服务器
      ! ssh git@gitlab.com
  4. 验证您的私有存储库,请检查此每个存储库部署密钥

  5. ! git@gitlab.com:{account}/{projectName}.git
    注意:要使用push,您必须授予对
    git服务器进行身份验证的公共ssh密钥的写入权限。


Sha*_*rad 5

You can almost use this link: https://qiita.com/Rowing0914/items/51a770925653c7c528f9

as a summary of the above link you should do this steps:

1- connect your google colab runtime to your Google Drive using this commands:

from google.colab import drive
drive.mount('/content/drive')
Run Code Online (Sandbox Code Playgroud)

It would need a authentication process. Do whatever it needs.

2- Set current directory the path you want to clone the Git project there:

in my example:

path_clone = "drive/My Drive/projects"
!cd path_clone
Run Code Online (Sandbox Code Playgroud)

don't forget to use ! in the beginning of cd command.

3- Clone the Git project:

!git clone <Git project URL address>
Run Code Online (Sandbox Code Playgroud)

now you would have the cloned Git project in projects folder in you Google Drive (which is also connected to your Google Colab runtime machine)

4- Go to your Google Drive (using browser or etc) and then go to the "projects" folder and open the .ipynb file that you want to use in Google Colab.

5- Now you have Google Colab runtime with the .ipynb that you wanted to use which is also connected to your Google Drive and all cloned git files are in the Colab runtime's storage.

Note:

1- Check that your Colab runtime is connected to Google Drive. If it's not connected, just repeat the step #1 above.

2- Double check by using "pwd" and "cd" commands that the current directory is related to the cloned git project in google Drive (step #2 above).


Vin*_*san 5

The very simple and easy way to clone your private github repo in Google colab is as below.

  1. Your password won't be exposed
  2. Though your password contains special character also it works
  3. Just run the below snippet in Colab cell and it will execute in an interactive way
import os
from getpass import getpass
import urllib

user = input('User name: ')
password = getpass('Password: ')
password = urllib.parse.quote(password) # your password is converted into url format
repo_name = input('Repo name: ')

cmd_string = 'git clone https://{0}:{1}@github.com/{0}/{2}.git'.format(user, password, repo_name)

os.system(cmd_string)
cmd_string, password = "", "" # removing the password from the variable
Run Code Online (Sandbox Code Playgroud)

  • @Ritwik 密码不会保存在笔记本上 (4认同)
  • 你说密码不会被泄露,但如果有人可以访问colab笔记本执行的bash语句,他们就可以轻松找到密码,不是吗?如果您假设没有人可以访问此类 bash 语句,那么其他方法究竟如何暴露密码呢? (2认同)
  • @ohailolcat 这假设没有人正在读取日志文件...... (2认同)