介绍:就是想实现简单的ajax上传数据,但是当数据量较大的时候,问题就一个一个接着来了,其实数据也不是很大,就是csv格式数据 不到5w条数据。大小5M,一开始认为这个很简单,就是先上传一下文件,然后读取一下,存到数据库就好了,结果,可能我比较菜,弄了半天做出这个功能。环境是linux.在整个调试过程下来,我觉找到问题点才是最重要的,但是一般当一段程序除了问题之后,我们都会觉得,这里有问题,那里有问题,其实这个时候,我们可以这样,一步一步的来测试,找到问题点在哪,这样处理问题来,就不会那样束手无策了--
遇到的问题就从先到后的一一说吧。
问题1 按照我最初的想法,先上传文件再读取文件。这里问题就来了,当文件较大的时候上传较慢,导致客户看到的操作一直处于等待状态,不人性化。
处理办法:我是这样做的,大神有更好的办法,求介绍。我先把文件上传上去,然后把文件存到一个特定的文件夹就叫 import吧 ,然后返回一个这个文件名字。这样就确保了文件是上传成功的。并且我可以在他返回名字的这一步用js 给客户一个提示。然后就是ajax去请求php读取文件,插入数据库。可是问题来了。
问题2 当我用ajax去请求php读取文件并插入数据库的时候,遇到一个问题,就是ajax请求总是在1min的时候,断掉。我一想 ,这应该是php的最大执行时间max_execution_time的原因吧,结果我修改为300秒。还是这样,那我就认为会不会是apache的 最大get时间max_input_time呢,我就在代码加一个 ini_set 结果,用ini_get 查看max_input_time,用ini_set设置无效,还是60秒,在网上查了很多资料,还是不知道为啥。有大神知道的,请给我回复下。菜鸟先谢过了。那没办法,我只能去服务器把php.ini配置修改了。经理说不让修改的,为了测试,偷偷改了--最后修改回来了。修改之后,测试,还是不行。还是到一分钟 就执行超时。真的很纳闷。不知道什么原因。求指教。那没办法。
这种办法行不通了,对一个5m的文件只能分行读取了。然后就是对代码的一通修改,分行读取是这样操作的,先ajax请求,然后每次读取2000条 然后对这2000条数据进行处理,插入数据库(文章最后介绍一个好用的分行读取函数)。然后每次ajax执行完,返回一个状态符,和本次读取到的行数,然后下次接着读。知道最后读取完。这中间还遇到一个问题:就是当我对每一行数据进行查重的时候遇到的,是这样的,我对得到的内容进行循环,然后查一下每行是否存在,当我判断$count是否大于0 的时候,当已存在的时候,我用continue,执行下一次循环。但是当我在导入10000条的时候,总是在8000条的时候报错说 服务器内部错误。很闷,不解问什么,结果只能用if else代替了。纳闷。一个小提醒:插入数据库的时候 不要一条一条的插入,最好这样 inset into aaa(`xx`,`xxx`)values('111','111'),('222','222')。这样 速度会快很多。
ip5hAAAgAElEQVR4nOzdPYvjyNrw8fogG68Vr9ONrc8wKFuL/gJmU0vpgGHpyNqsMR10aIXNJNsziTk02IGPafA2e8DJw4E7OXDfwUn2CUovVaUqWfLYlu3+Nz+GHlkuyZK6+7rqVaTzHQAAAAAUROdnAAAAAOCikCQAAAAA0JAkAAAAANCQJAAAAADQkCTgsiSTlTf5q/PTgOHa78u1nz8AAGdGknCtosEXITL+uLPTSIZfv+cEwl4/nCofKn4Vdxk/0T+v46UofjWCv2SykrsRFB6L9eKHo1dxtwofuz+9w86/raM9V9PY68WdXxMAAOqRJFyn6dLrLbs/jfkune+S4dcDk4RxIAaz8r/JWsRb+X0U68Hc45s3emt3VpdUcxyO1tElldOO+76Eo/MnCTNfCG/47Sjnf4CjPFfRQHSY2AMA0ARJwvVR2xCE+CKUbCHsFdsXkb7RG/4pa/3Ll6ZLz1ZIOl4UhXvDP+2H7i3D3lfZCJAMv/rDoqhF8yg27GmhUjJZWQM4tRpY3L2KPFuor9m1BHOPb16lkD2XOi73VwPicFScTxa1y5Px47WxXTuoUZNtO59DynFx38fyOel93Zttuu5LmicJxY3IztZ6nbON2TXM3uK+C8mwLypfRW4Q9oQQWhtU2/Mv7qw3+StN1trZFv+1PVrGc5Xdr6Qss3y15nmjMQEAcPFIEq6TrSUh7Cmx4HhhifvllvFChuZFlJ/KXkPF/uONmmAUcXw0+CIGm3J/USYJ6vdGPOo280WgZxRbX4kjNe6WBFfNbnW7GuUnk9XePCGKX4sa6GSyKk4sHCmBYLLW4/syZ1CPbm0BcJ1P23KcHPdRvUfRQM8P7Zz3RUkSyhNzXudkffhnqZrGnhCiUahtP3/t2iqtDWmyVT6LmYxVnystCVE+Y+3zNvMbJzkAAHSCJOE6VZOE6dLLI3gpGpQ5QDpemLG72oxgNALoL+XB5cZ3RJNad6PqgZwfIfbMJGGXzsv614bdjZomCZWaeDWutdn61iM+vnlFNCmvc1zWjluDxdQaELvPp1059Q+J5T7u0vnGr970vWz3JWtRUS9U7XUuz1//XLaHqq4lwditUe8jy/lv/fzctH5T+kc4MEnY97wZzWgAAFwakoTr9P1JgjPo3/hCa2E4W5JQRNvpvBJEHqElwRH0O504SXCfz5GSBNd91BktTjY19yUcyZ5R6lnVXuf87UcZXJEM+01aEmrOP7vUajPCfOvfaS0Ah7Yk7HneSBIAABeOJOE62bsbKVmBEdPbYvdoYHZVN0ueLj2hdTdS9t/4+UsHJgmV7kb62AM9iDxGd6PqPEj19P23fl6PRI/WLWPB2uA+f0tShtSu82lbzv4nRL+P2nMyXgg9t7ReB2d1uDwf/WTqr3M4Wkf7mhH2a9HXqPa5mm99fbSJ9qQ9vnnNWhLU1oPi1drrQHcjAMClI0m4Psro5C9ClOMEnN2E9G5Fav2lXlTW8yQf3/xFiK/+QJvhVN2/yBDK/+YjZRvmCdbK1GrdrTJK+FXcFeMEZEfzavzn2l4tan9ltrq/fcBxvr0YvJvVTBvDXpPKQGTH+RxWjpXrPlpv+l6O+7IKH/VBwHuv8970Zo+ZL1qMWq45/2K70JuGlHHYKz8uxiW7n6vyeViFk3Wj68DAZQDAxSNJQHeMKVDnu7R2Lh106Gj35fubEbo9/2NgClQAwOUjSUCXjMXUcMOOsqLZLaAZAQBwDUgSAAAAAGhIEvCBWeapbNrjHwAA4IaRJAAAAADQkCQAAAAA0JAkAAAAANCQJAAAAADQkCRcrWnsCeENv3V/JgAAALgtJAnXKhn2yRAAAABwCiQJ14okAQAAACdCknCtSBIAAABwIiQJVykaCNGLOz8NAAAA3CSShGtFSwIAAABOhCThWpEkAAAA4ERIEq4VSQIAAABOhCTharFOAgAAAE6DJAEAAACAhiQBAAAAgIYkAQAAAICGJAEAAACAhiQBAAAAgIYkAQAAAICGJAF2yWTlTf7q/DTwQfC8AQBwUUgSYBHFr+Iu4ye7dL5Lk3WxRdy9itHb3kKSyUrufJTgL4prytn6zU7pWiS/98X9rM1bZn4svN+/tT1Q2OuH0+6Pa3neDrhoR33ekmFfCCGEfn2GX4X4IsQXMdhoF0EI0W7RktlSiPderG0cB+9CvA9mz8P+uxAbrTTb/tPYM7bUbwcAoA2SBFQkaxFv5fdRXAZtal1vbciuOVcN8da/oSQhuhctg/WDjAMx0I7SzXEdz9thjvW8WVY0Hy9Eb5md5+CLP963f61FzxL0b2RuMA7ehViO9+0/30UDYZxG/XYAAJojSYApmaysgZoWfj2+efE2zatv5f6yPtgI0SxB2+Obd5QWCaUcP7nQJCGvkNa+aqPJmR8Locqi9nz7/Sy6ly/1w6ddKqv/tT136VPsxULEgX8vxOc4kf/9HCf6scKeGkp2dVz781a0LXiTv8pWLHmLlUataj5gPG91z2ftc1gN+qPB17JVYbr0tMaEg5IEPUOTScJyLJsU+ovpvv3nBzQmyEYPrXkEAAArkgRUbf27V3G3Ch+17Wr4FY60FoYyyEvWe5OEcFSWnExWDbsJVcrZ+uUZ3lp3I1eNfh6mB1Gla5D5lodAxMJ/2EUyXp/P/Fj4D2ppM18E0UUcd//zprY2pMm2eLv6HDqeE+fzWf8c7k8S8lYF1/71XgZGh6JdOp8tZW4wjTcieNm/v7yY1oi/ZjtJAgCgEZIEOOT1rGoyYK3BbZckqNW3mXXU4HzMcowDPb55F5kktG9J2KV7gnUzsre/5SGQe+bbK8H6NPbaJQknPK76YChB/9bPnw01oDceoQOThH3P4amThGMxmmX2bgcAoCGSBJiiWAnIlFi8DL+StRpRtWxJOLBf0JUmCQfegppg3TFm4ORJwsmO63re0uLRUpsRtBYkS1elxi0Je55DW5KgjEP47u5Gx0KSAAA4EZIEmNSpZoxkQE0Yirit3P745jUYk9B80HNtOWU1c3bON5QkKF16voWfy+mDjhms27obdXJc1/OW3eXRm9mMUNzoxzevWUuC9fmsfw4tQf906Q//zN7bcODyOBBCiOpYgqNp2d1Ing9zHwEAGiBJgJ1ZR2sMHp3vwlH+fdlzYxVO1nlnJNnR3B7/haPm3Y3c5aizssbr2xqWoAwjLgb+PgTFqGJ91lF1zHE/fCoGEAvxOQ7v5f6xHwsR99Wj2Cqbuzpu3XB5pRkh35I/b35cjEt2Pyf253PPc2gN+sPeF9sUqM79o8GJBwC0HLgcDYSw9vUCAKCCJAF2rqANt6MyBWqHx720561t9yFHUtFq8YTWWk6B6mp2AADAgiQB+LiMRc1u/rjNWRdTc3aspuYaon0sLKYGADglkgQAAAAAGpIEXADLfJQtZkcFAADAcZEkAAAAANCQJAAAAADQkCQAAAAA0JAkwK66KBVwOjxvAABcFJIEWKiL4Gaz16srlzVbtqxY9Irgry1l5eOGZr650lkjxlSkXR3X8rwdcNGO+rxZp0CNBl+E+OLl6y6XF8E6BWrtdVsK8W5MVDoO3oV4H8yeh/13ITZaabb9mQIVAHBKJAmoSNbFGrdRXAZtal1vFDcNxaghPkB0L1oG6wepLGrWzXEdz9thjvW8VRdHS4ZfxWCTjheVJMG+f71FzxL0b2RuMA7ehViO9+3fejE1AABaIEmAybX2rRZ+Pb558TbNq2/l/rI+2AjRLEGbOuHpvhaJrPx4bZkUVWncyA4hSx6tfLlntkP+ljbHPdrFzCqkta/aaHLmx0Kosqg9334/i+7lS/3waZfK6n9tz136FHuxEHHg3wvxOU7kfz/HiX6ssKeGkl0d1/68FW0L3uSv8kbLu1a97+7nre75rH0enEH/EZMEY9npabyRucE4eBf9xXTf/vMDGhNko8elr2QHALgEJAmo2vp3r+JuFT5q29XwKxxpLQxlkJes9yYJ4agsOZmsmuUJWaCvlZZso7LM/Hwe3zwZFCZr+a7i9Noet0OuGv08TA+iStcg8y0PgYiF/7CLZLw+n/mx8B/U0mZ+ZT3gjo67/3lTWxvs9939vLmez/rn4dRJwsvA6FC0S+ezpcwNpvFGBC/795cX0xrx12wnSQAANEKSAIe8nlVNBqw1uO2SBMu6aXtWTHOWrxdVJglFfXPe1uEnhxz3KNq3JOzSPcG6Gdnb3/IQyD3z7ZVgfRp77ZKEEx7X+ryl862f3yM1oLffd9fz5np+9j0Pp04SjsVoltm7HQCAhkgSYIpiJSBTgvIy/Mor6YvtbVoStn7LKnxH+Vv/TqsJ3pMktD9ul7egJlh3jBk4eZJwsuO6nrfy3qnNCK77rrylWUvCnueBJAEA8MGRJMCkTjVjJANqwlDEbeX2xzevwZiE5oOeixLsNcFFkFd0MZrXJAmtj9shpUvPt/BzOX3QMYN1W3ejTo7ret7S+U6G8mYzgvW+u5831/NZ/zwcJ0kYB0IIUR1LcDQtuxvJ82HuIwBAAyQJsDPraI3Bo/NdOMq/L3turMLJOu+MJDua2+O/cNS020/RxykfZlB2dlK6P638eCXuXv1JfibxVu7sTf5Sh642P27XlGHExcDfh6AYVazPOqqOOe6HT8UAYiE+x+G93D/2YyHivnoUW2VzV8etGy6vNCNoj0R532VLket5sz+fe56HatAv5z8t9ZZp7f7pfBcNTjwAoOXA5WgghLWvFwAAFSQJsHMFbbgdlSlQOzzupT1vbbsPWfcPe60WT2it5RSormYHAAAsSBKAj8tY1Ozmj9ucdTE1B8diaq4h2sfCYmoAgFMiSQAAAACgIUnABbDMR3n5wwYAAABuFkkCAAAAAA1JAgAAAAANSQIAAAAADUkCAAAAAA1JAgAAAAANSQIAAAAADUkCAAAAAA1JAgAAAAANSQIAAAAADUkCAAAAAA1JAgAAAAANSQIAAAAADUkCAAAAAA1JAgAAAAANSQIAAAAADUkCAAAAAA1JAtCBP1p+dX7CAADgQyFJADpAkgAAAC4ZSQLQAZIEAABwyUgSYJdMVt7kr85P40o9D/ub4beaHf7447d///DDv3/rOEnYe54AAOBjIkmARRS/iruMn3R/PpdnthTivRe7Xw1e9I3Pw/67EO+iv5ju0vnujz/++OenH/7++dd/2JOCp3/9rL2aznfpOHgX4n0wk0Upwf1sKYS+pfmnMM6z/nMBAICPgiQBFclaxFv5fRR/V5IQjtZRm+1XZNFzBtPW6nlj4x9//PHH06//625M+OenH/7+VL6WznfpNN7ITGAcvAuxHO8/aDrfpePAlTxY31LzuQAAwMdBkgBTMllVE4Nw9CruVuFknbcwrMLH/NWk2Pha9lB6fPPuyuaIskXCtd14afSmH/rVm/yVTFbi7lXcraP5Tn7vx8WhtaxDvkXcvYrRylOKsnzYYV9UvrwGVfKLnngfzGwvWZoRUmuSUNuY8I9ff/7fX5+qScJyLJsUshYJV/klZ5JgP0/H55r5QgjRD6fVcgAAwA0iSUDV1jfSgPkuzSLvPBZ/fPOK75NtpOyjJhitWhLCUXnEZLJS84QsD5FbkrU8RDJZFeejjqBQv49iLd84opeBvXuPK1i3Jgl//Pbpb1djwm+fzCRhPlvK3GAab2zdmVolCa79HZ+LJAEAgI+FJAEOeb1+EfQbCUDZ4KA3DhyYJFRaGLTGgWRdHUWttXhoO8gkx9LCUHVwS4KDvXo+dSUJcuzBp/3jl/ce2gz6p/FGiHed0kPJeZ4AAAApSQKqoljrSlQE32pNf1rG6Fv/TmsBOLQlYevXVPm3SxK0N56oJcGqZrIgR5IgGxN+/teTOz84LEko2FoSmNQIAADUI0mASZ3aSK2JD0fGUIF1NN+lj29lp//HN89sScjzh2StF2XZHsWvzklX2yQJWjKjDMI+vbrqeWeS0KwxYe/R2yQJ7ZsRxoEQQjCgGQCAD4MkAXbV4cvhaOXHq+oY5Xw88au4y3ZQY3d7tx/H9nLAcfmS2neo7M5UHNRPytLkKdkKOYf66nl3ktCoMeHwo1eShAOaEaKBECK49gmpAABAcyQJsLMmCcZQZijyUcWOHeqShAaNCXtPoHHov+c8rW/xGbIMAMAHQ5KARpRJRc/Xxf+WVBdTqzQmfPrngUnCwYupAQAA2JEkAB2o61p0UEsCAADAEZEkAB0gSQAAAJeMJAEAAACAhiQBAAAAgIYkAQAAAICGJAF2yWTlXNoMuBU851xnAIAVSQIs1EWXs9USyuXPmk6EWqx3dpTgoG49Zrng2gXMzZr83hf3szZvmfmx8H7/1vZAYe92Fi5Ifu+LWIhYtLx0hzCum+U5/4DGwbsQ74OZnKW3nEjXtb296/59Mo0961rjru0AcCtIElCRrEW8ld9HcRk8qXWBtX9iNeeqQdz6F5AkRPfniHTTcSAGpz/K7V0647o5nvMDhCP7wt6u7ZdlGm9kDjAO3oVYjvdtb+v6f59EA+HbPr5rOwDcBpIEmKprLRfbyz/Pj29evE3z6j25v6wvNP6EW/6oP755R6lBVMrxkyMnCcmwLypfXl1l6syX1eGFLOTNt9/Ponv5Uj982qXWGvSn2IuFiAP/XojPcSL/+zlO9GOFvTI08WIhYuE/fAs/y8KDyDifynH1Uw3KKDY7uhBxP/w90N/i/rxNys9Ltp1nppok5MUKYV6EopBmn8t23VzPeTh6FXercFLUcyurjCuV39oPglojXtSUu7
新闻热点
疑难解答