Standard URL Normalization - Java

Question

I would like to ask if there's any Java package or library that have the standard URL normalization?

5 Components of URL Representation

http://www[dot]example[dot]com:8040/folder/exist?name=sky#head

1. scheme: http
2. authority: www.example.com:8040
3. path: /folder/exist
4. query: ?name=sky
5. fragment: #head

The 3 types of standard URL normalization

Syntax-Based Normalization

• Case normalization – convert all letter at scheme and authority components to lower case
• Percent-encoded normalization – decode any percent-encoded octet that corresponds to unreserved character, such as %2D for hyphen and %5 for underscore
• Path segment normalization – remove dot-segments from the path component, such as ‘.’ and ‘..’

Scheme-Based Normalization

• Add trailing ‘/’ after the authority component of URL
• Remove default port number, such as 80 for http scheme
• Truncate the fragment of URL

Protocol-Based Normalization

• Only appropriate when the results of accessing the resources are equivalent
• For example, example.com/data is directed to example.com/data/ by origin server

Answer 1

正如其他人所提到的,java.net.URL和/或java.net.URI是一些明显的起点.

这里有一些其他选择:

1. Galimatias(西班牙语中的"乱码")似乎是一个自以为是且相对流行的Java URL规范化库.源代码可以在github.com/smola/galimatias找到.

   galimatias开始时对java.net.URL和java.net.URI感到沮丧.它们都适用于基本用例,但对其他用户则严重破坏

2. 该github.com/sentric/url-normalization库提供它颠倒了域部分的另一个(不寻常的,在我看来)的方式; 例如"com.stackoverflow"而不是"stackoverflow.com".

您可以在Github上找到其他变体,有时用Python,Ruby和PHP等语言实现.

Answer 2

URI uri = URI.create("http://www.example.com:8040/folder/exist?name=sky#head");
String scheme = uri.getScheme();
String authority = uri.getAuthority();
// ...

https://docs.oracle.com/javase/1.5.0/docs/api/java/net/URI.html