Perl:提取域名

Question

提取URL的域名

还有一个解析URL的请求,但我发现了许多不完整或理论上的例子.我想肯定有一些在perl中工作的东西.

我有以下网址:

https://vimdoc.sourceforge.net/htmldoc/pattern.html
http://linksyssmartwifi.com/ui/1.0.1.1001/dynamic/login.html
http://www.catonmat.net/download/perl1line.txt
https://github.com/robbyrussell/oh-my-zsh/wiki/Cheatsheet
https://drive.google.com/drive/u/0/folders/0B5jNDUmF2eUJuSnM
http://www.gnu.org/software/coreutils/manual/coreutils.html
http://www.catonmat.net/download/perl1line.txt
https://feedly.com/i/my
http://vimhelp.appspot.com/
https://git-scm.com/doc
https://read.amazon.com/
https://github.com/netsamir/following
https://scotch.io/
https://servicios.dgi.gub.uy/
https://sourcemaking.com/
https://stackedit.io/editor
https://stripe.com/be
https://toolbelt.heroku.com/
https://training.github.com/
https://vimeo.com/54505525
https://vimeo.com/tag:drew+neil
https://web.whatsapp.com/
https://www.ctan.org/
https://www.eff.org/
https://www.mybeluga.com/
https://www.solveforx.com/
https://www.symynd.com/
https://www.symynd.com/#
https://www.tizen.org/
http://workforall.net/CDS-Credit-default-Swaps.html#Credit_Default_Swaps_CDS

尝试仅提取域名.例如:

linksyssmartwifi.com
amazon.com
github.com

我曾尝试使用Perl和Vim,但无法完成任务.我的最佳近似值如下

 perl -pe 's!(^https?\://.*[\.](.+\..+?)/.*$)!$1 -- [$2] !g' all_urls_sorted.txt

其中一些被正确解析(见[]),其他不是:

   https://sites.google.com/site/steveyegge2/singleton-considered-stupid -- [google.com] 
https://sourcemaking.com/
https://stackedit.io/editor
https://stripe.com/be
https://toolbelt.heroku.com/ -- [heroku.com] 
https://training.github.com/ -- [github.com] 
https://vimeo.com/54505525
https://vimeo.com/tag:drew+neil
https://web.whatsapp.com/ -- [whatsapp.com] 
https://wiki.haskell.org/GHC -- [haskell.org] 

正如我的测试显示的那样,直接从//(以https？://开头)的URL被排除在外.

如果你知道如何解决这个问题,我会非常高兴.

谢谢

Answer 1

使用URI模块:

#!/usr/bin/env perl

use strict;
use warnings;
use v5.10;

use URI;

while (<DATA>) {
    chomp;
    my $uri = URI->new($_);
    my $host = $uri->host;
    my ($domain) = $host =~ m/([^.]+\.[^.]+$)/;
    say $domain;
}

__DATA__
https://vimdoc.sourceforge.net/htmldoc/pattern.html
http://linksyssmartwifi.com/ui/1.0.1.1001/dynamic/login.html
http://www.catonmat.net/download/perl1line.txt
https://github.com/robbyrussell/oh-my-zsh/wiki/Cheatsheet
https://drive.google.com/drive/u/0/folders/0B5jNDUmF2eUJuSnM
http://www.gnu.org/software/coreutils/manual/coreutils.html
http://www.catonmat.net/download/perl1line.txt
https://feedly.com/i/my
http://vimhelp.appspot.com/
https://git-scm.com/doc
https://read.amazon.com/
https://github.com/netsamir/following
https://scotch.io/
https://servicios.dgi.gub.uy/
https://sourcemaking.com/
https://stackedit.io/editor
https://stripe.com/be
https://toolbelt.heroku.com/
https://training.github.com/
https://vimeo.com/54505525
https://vimeo.com/tag:drew+neil
https://web.whatsapp.com/
https://www.ctan.org/
https://www.eff.org/
https://www.mybeluga.com/
https://www.solveforx.com/
https://www.symynd.com/
https://www.symynd.com/#
https://www.tizen.org/
http://workforall.net/CDS-Credit-default-Swaps.html#Credit_Default_Swaps_CDS

输出:

sourceforge.net
linksyssmartwifi.com
catonmat.net
github.com
google.com
gnu.org
catonmat.net
feedly.com
appspot.com
git-scm.com
amazon.com
github.com
scotch.io
gub.uy
sourcemaking.com
stackedit.io
stripe.com
heroku.com
github.com
vimeo.com
vimeo.com
whatsapp.com
ctan.org
eff.org
mybeluga.com
solveforx.com
symynd.com
symynd.com
tizen.org
workforall.net