将1000000个文档插入RavenDB

Question

我想在RavenDB中插入1000000个文档.

class Program
{
        private static string serverName;
        private static string databaseName;

        private static DocumentStore documentstore;
        private static IDocumentSession _session;

        static void Main(string[] args)
        {

            Console.WriteLine("Start...");

            serverName = ConfigurationManager.AppSettings["ServerName"];
            databaseName = ConfigurationManager.AppSettings["Database"];

            documentstore = new DocumentStore { Url = serverName };
            documentstore.Initialize();

            Console.WriteLine("Initial Databse...");

            _session = documentstore.OpenSession(databaseName);

            for (int i = 0; i < 1000000; i++)
            {
                var person = new Person()    
                {
                    Fname = "Meysam" + i,
                    Lname = " Savameri" + i,
                    Bdate = DateTime.Now,
                    Salary = 6001 + i,
                    Address = "BITS provides one foreground and three background priority levels that" +
                              "you can use to prioritize transBfer jobs. Higher priority jobs preempt"+
                              "lower priority jobs. Jobs at the same priority level share transfer time,"+
                              "which prevents a large job from blocking small jobs in the transfer"+
                              "queue. Lower priority jobs do not receive transfer time until all the "+
                              "higher priority jobs are complete or in an error state. Background"+
                              "transfers are optimal because BITS uses idle network bandwidth to"+
                              "transfer the files. BITS increases or decreases the rate at which files "+
                              "are transferred based on the amount of idle network bandwidth that is"+
                              "available. If a network application begins to consume more bandwidth,"+
                              "BITS decreases its transfer rate to preserve the user's interactive"+
                              "experience. BITS supports multiple foreground jobs and one background"+
                              "transfer job at the same time.",
                    Email = "Meysam" + i + "@hotmail.com",
                };

                _session.Store(person);

                Console.ForegroundColor = ConsoleColor.Green;
                Console.WriteLine("Count:" + i);
                Console.ForegroundColor = ConsoleColor.White;
            }

            Console.WriteLine("Commit...");

            _session.SaveChanges();
            documentstore.Dispose();

            _session.Dispose();

            Console.WriteLine("Complete...");
            Console.ReadLine();
        }
    }

但是会话没有保存更改,我收到一个错误:

mscorlib.dll中发生了未处理的"System.OutOfMemoryException"类型异常

Answer 1

一个文件会被用来处理少量的请求.相反,尝试以1024的批量插入.之后,处理会话并创建一个新会话.您获得的原因OutOfMemoryException是因为文档会话缓存所有组成对象以提供工作单元,这就是您在插入批处理后应该处理会话的原因.

一个巧妙的方法是使用Batch linq扩展:

foreach (var batch in Enumerable.Range(1, 1000000)
 .Select(i => new Person { /* set properties */ })
 .Batch(1024))
{
 using (var session = documentstore.OpenSession())
 {
   foreach (var person in batch)
   {
     session.Store(person);
   }
   session.SaveChanges();
 }
}

两者的实现Enumerable.Range和Batch懒惰,不要让所有的对象在内存中.