firefox缓存哈希密钥生成算法bug

Question

Firefox中存在一个错误(即使在新的测试版和雷区版本中),由于在缓存哈希中创建密钥的算法,它会阻止某些文件的缓存. 这是该函数源代码的链接.

我想确保我的所有网站文件都可以缓存.但是,我不明白为什么他们的散列函数无法为不同的URL创建唯一键.我希望有人可以在psuedo-code或java中描述这个mal函数.

最好为开发人员创建一个实用程序,以确保在修复此错误之前使用唯一的URL.

---

编辑:有一些非常有用的答案,但是,我需要更多的逐步帮助来创建一个实用程序来检查这些缓存混淆.获得一些可以重现firefox创建的密钥的java代码会很棒.因此,在这个问题上开启了赏金.

---

编辑2:这是一个部分工作的Java端口(使用处理编写).注意底部的测试; 前三个按预期工作,但其他人没有.我怀疑有关签名/未签名的内容.建议？

//
// the bad collision function
// http://mxr.mozilla.org/mozilla/source/netwerk/cache/src/nsDiskCacheDevice.cpp#240
//

//248 PLDHashNumber
//249 nsDiskCache::Hash(const char * key)
//250 {
//251     PLDHashNumber h = 0;
//252     for (const PRUint8* s = (PRUint8*) key; *s != '\0'; ++s)
//253         h = PR_ROTATE_LEFT32(h, 4) ^ *s;
//254     return (h == 0 ? ULONG_MAX : h);
//255 }

//
//  a java port...
//

String getHash( String url )
{

//get the char array for the url string
char[] cs = getCharArray( url );

int h = 0;

//for (const PRUint8* s = (PRUint8*) key; *s != '\0'; ++s)
for ( int i=0; i < cs.length; i++ )
{  h = PR_ROTATE_LEFT32(h, 4) ^ cs[i];
}

//looks like the examples above return something in hex.
//if we get matching ints, that is ok by me.
//but for fun, lets try to hex the return vals?
String hexVal = hex( h );
return hexVal;
}

char[] getCharArray( String s )
{
  char[] cs = new char[s.length()];
  for (int i=0; i<s.length(); i++)
  { 
    char c = s.charAt(i);
    cs[i] = c;
  } 

  return cs;
}

//
// how to PR_ROTATE_LEFT32
//

//110 /*
//111 ** Macros for rotate left and right. The argument 'a' must be an unsigned
//112 ** 32-bit integer type such as PRUint32.
//113 **
//114 ** There is no rotate operation in the C Language, so the construct
//115 ** (a << 4) | (a >> 28) is frequently used instead. Most compilers convert
//116 ** this to a rotate instruction, but MSVC doesn't without a little help.
//117 ** To get MSVC to generate a rotate instruction, we have to use the _rotl
//118 ** or _rotr intrinsic and use a pragma to make it inline.
//119 **
//120 ** Note: MSVC in VS2005 will do an inline rotate instruction on the above
//121 ** construct.
//122 */
//...
//128 #define PR_ROTATE_LEFT32(a, bits) _rotl(a, bits)


//return an int (32 bit).  what do we do with the 'bits' parameter?  ignore?
int PR_ROTATE_LEFT32( int a, int bits )
{    return (a << 4) | (a >> (32-bits)); 
}

//
// examples of some colliding hashes
// https://bugzilla.mozilla.org/show_bug.cgi?id=290032#c5
//

//$ ./hashit "ABA/xxx.aba"
//8ffac222
//$ ./hashit "XyZ/xxx.xYz"
//8ffac222
//$ ./hashit "CSS/xxx.css"
//8ffac222
//$ ./hashit "JPG/xxx.jpg"
//8ffac222

//$ ./hashit modules_newsfeeds/MenuBar/MenuBar.css
//15c23729
//$ ./hashit modules_newsfeeds/ListBar/ListBar.css
//15c23729

//$ ./hashit modules_newsfeeds/MenuBar/MenuBar.js
//a15c23e5
//$ ./hashit modules_newsfeeds/ListBar/ListBar.js
//a15c23e5



//
// our attempt at porting this algorithm to java...
//

void setup( )
{

String a = "ABA/xxx.aba";
String b = "CSS/xxx.css";
String c = "CSS/xxx.css";
String d = "JPG/xxx.jpg";

println( getHash(a) ); //yes 8ffac222
println( getHash(b) ); //yes 8ffac222
println( getHash(c) ); //yes 8ffac222
println( getHash(d) ); //no [??] FFFFFF98, not 8ffac222

println( "-----" );

String e = "modules_newsfeeds/MenuBar/MenuBar.css";
String f = "modules_newsfeeds/ListBar/ListBar.css";

println( getHash(e) ); //no [??] FFFFFF8C, not 15c23729
println( getHash(f) ); //no [??] FFFFFF8C, not 15c23729

println( "-----" );

String g = "modules_newsfeeds/MenuBar/MenuBar.js";
String h = "modules_newsfeeds/ListBar/ListBar.js";

println( getHash(g) ); //yes [??] FFFFFF8C, not a15c23e5
println( getHash(h) ); //yes [??] FFFFFF8C, not a15c23e5

}

Answer 1

根据我对读取bugzilla条目的理解,当出现两个不同的问题时,错误就会出现:

1. 他们的哈希算法为"足够相似"的网址生成冲突.从"类似的足够"这个错误来看似乎意味着每4个字符(或者可能是8个)的网址是相同的,并且
2. 他们处理哈希冲突的逻辑失败了,因为他们还没有将具有相同哈希值的前一个URL刷新到磁盘.

所以基本上,如果你有一个页面有两个非常相似的网址,这可能会发生在某些版本的Firefox上.我通常不会在不同的页面上发生这种情况,因为那时FF将有时间将条目刷新到磁盘以避免计时问题.

因此,如果您有多个资源(脚本,图像等)都是从同一页面加载的,请确保它们具有完全不同的9个字符.确保这一点的一种方法是使用随机数据附加一个查询字符串(您忽略),例如:

• http://foo.com/resource.js?r = dn#@ JdsK#

Answer 2

以下是算法的工作原理:

initialize hash to 0
for each byte
    shift hash 4 bits to left (with rotate)
    hash = hash XOR character

视觉(16位版本):

00110000             = '0'
    00110001         = '1'
        00110010     = '2'
            00110011 = '3'
0100            0011 = '4'
00110101             = '5'
====================
01000110001000010000  (and then this will be 'rotated'
                       so that it lines up with the end)
giving:
        00100001000001000110

这意味着如果你有相同长度的字符串并且大多数相同,那么在至少一种情况下,char的低4位和下一个char xor的高4位必须是唯一的.然而,粘附的32比特数到表可能是以往弱的,这意味着它需要在字符串中的特定位置的lower4 XOR upper4(模8个字符)的方法是唯一的.