The URL can be used to link to this page

Home My WebLink About5. Information Technology Storage Discussion 10/14/2014Village of Mount Prospect Mount Prospect, Illinois INTEROFFICE MEMORANDUM TO: ASSISTANT VILLAGE MANAGER DAVE STRAHL FROM: IT DIRECTOR DATE: OCTOBER 3, 2014 SUBJECT: VILLAGE DATA STORAGE At the September 16th Village Board meeting, trustees ratified the purchase of two Tegile SANs (storage area networks) for $82,600. However there were questions regarding the urgency of the purchase and the need to expand data storage. I have reviewed the questions posed at the board meeting and provided answers below. Questions: • What data is maintained in the cloud? The Village has some cloud -based applications such as Fire Department rosters, Microsoft Project for Public Works, and archived email. However, the Village has not adopted a philosophy of keeping general data files or active email in the cloud, nor any resident information. • What data is maintained on the SANs (storage area networks)? Previously we had traditional physical servers that were self-contained in that the memory, processing power, and storage are all housed on the same physical machine. The servers were replaced every four or five years with new servers that had larger hard drives than the previous ones, so storage growth has not been a significant issue in the past. Now we have virtual servers that share memory and processing power from a physical host server, and share storage from a SAN. A pool of 7 host servers (4 at Village Hall and 3 at the EOC) provide the memory and processing power. A pool of 5 SANs (3 at Village Hall and 2 at the EOC) provide the storage for the files that make up the virtual machines. The Village has 31 virtual servers that share SAN storage. This includes 5 file servers, 16 application and database servers including ERP, 9 utility servers, and the document imaging server. Any of the 31 virtual servers can be moved, either automatically or manually, between hosts or storage as needed. The application servers will be the prime candidates to be moved from the old storage to benefit from the newer faster technology. What kind of disaster recovery grocedures are in place for data stored on the SANs? Data is replicated bi-directionally every day between the data centers in Village Hall and in the EOC. Additionally, backups are created daily and replicated between data centers. Village Data Storage October 3, 2014 Page 2 Could we have planned better for data expansion? New storage was pre -funded through the replacement schedule for 2014 to replace the existing five-year-old Equallogic SANs. Replacement schedule purchases do not show in the Computer CIP as new projects since they are simply replacements. The cost of the new Tegile SANs exactly matches the amount on the replacement schedule so only the installation and support are new costs. Early in 2014, staff began evaluating storage products for the purchase. Staff started collecting data on storage growth a couple of years ago for proper planning and had been carefully monitoring available space. There was sufficient space to expand storage on the virtual servers as needed. Storage on each of the SANs is broken up by functions, with space delegated for basic storage, and space delegated for replication of storage from the partnering SAN in the other location. As basic storage usage grew, the partner replication space needed to grow as well. We hit a point where there was not enough replication space on the partner SAN for all of the data to replicate and we could not reconfigure it to do so. Was there some event that created a large amount of data? There was no single event requiring the need for more storage. The issue became more critical when the Police Department could no longer save files to their file server. We could easily give them additional space but knew that if we did so, the server replication would fail due to limited replication space on the partner SAN. We gave the Police Department a choice between "breaking replication" or cleaning out unnecessary files to get them by. They chose the latter until new storage was purchased. IT staff then provided an analysis of the Police file server usage and offered some suggestions to Police. They use roaming profiles for all their users so that their users have a consistent desktop experience no matter which PC they log into. A couple of those users had a significant amount of data in recycle bins which bloated their profiles using space needlessly. They have since emptied those recycle bins and cleaned up some files, and currently have over 40 GB of free space available. A couple of weeks later, we hit a critical point with replication space that required us to stop replicating one of the storage volumes. This was caused by the natural growth of the servers in general. We chose to stop replicating the volume that holds the Village Hall server. By that time we were well prepared to take the purchase to the Board on September 2nd However the meeting was canceled. • What increases have we seen in data storage? The largest increase has been on our Laserfiche Document Imaging Server where data usage has increased by 229 gigabytes or 70% in the last 15 months. The chart on the next page shows storage use by department. Village Data Storage October 3, 2014 Page 3 Document Imaging Storage 600 Clerk 22 Community Development 500 176 Finance 400 20 a Fire* 00 300 18 154 127 ■Human Resources** 146 60 ■ Police N ZOO 59 '^ 21 n Public Works 100 --hfi--' RF 170, *3 GB on each date -too I small to show on chart 0 ** 1/2 GB on each date - 6/15/13 11/1/13 9/29/14 too small to show on chart General file storage has not grown as much thanks to staff efforts on Records Days. Information Technology provides a statistical analysis of file storage usage to each department before and after each Records Day. The analysis includes reports by user, file age, file type, and storage space used. The chart below page shows a history of data storage by file server. Data Storage by File Server 1400 1200 - 256 - 247 ---177 -� S, 1000 —.193 _ 194 m 800 280 264 350 354 318 Q 600 366 351. 350 350 395 N 400 ' 20028' 328 354 328 348 0 m 1167 GB 1137 GB 1310 GB 1279 GB 1238 GB before after before after as of Records Records Records Records Oct. 3, Day Feb. Day Feb. Day Feb. Day Feb. 2014 2013 2013 2014 2014 Fire Police Public Works Village Hall Newer technologies and Village procedures are trending from paper to digital formats so we can expect data storage to continue to grow. For example: • Finance is creating electronic payroll, financial, and billing reports instead of paper. • Graphic files are getting bigger since newer cameras provide higher resolution and more colors. Village Data Storage October 3, 2014 Page 4 • Several of the departments have annual initiatives to scan documents to our Laserfiche document imaging system. Laserfiche is primarily used for historical documents or other documents that must be kept per the Local Records Act because it provides quick, comprehensive document search capabilities. The new Tegile storage units will each provide a minimum of 9.7 TB of usable storage compared to the Equallogic units that provide 6.28 TB each. Also the compression and deduplication features of the Tegile hardware will allow more data to be stored in less space. Two other existing Equallogic units are available for replacement in 2015 and the last one in 2016. We will determine next year if they need to be replaced at that time or if the purchases can be postponed. To help predict storage growth in the future, staff will start charting data on the space used on the SANs. Shared SAN storage will continue to be the primary focus for future storage purchases. • Could this purchases have gone to bid? Many vendors, including Tegile, ask that we choose a reseller. Once our selection is made they work with the reseller to provide a quote. We tried to get a second quote from a different reseller for comparison but Tegile would not work with the second vendor since a quote had already been provided to us. Other Scheduled Purchases for 2014 For Police, there are five new desktop PCs and ten ruggedized laptops that are budgeted but not yet purchased because they are not ready for them. Also IT will request a scheduled replacement for the Village web server. This will be a traditional physical server since the web server resides in the DMZ (the "neutral zone" between the Village's internal network and the Internet), and there is a firewall between the web server and the virtual infrastructure. A security assessment is also planned to span from this year into next. There has been some planning and discussion, but staff is still working to define the scope of this project. Conclusion I hope this answers all of your questions. As explained above, this purchase was planned and only became critical because of concerns that replication could no longer work as it did in the past due to space constraints on the partnering SAN. Staff will continue to collect predictive data for future storage growth where possible. ?"Joan Middleton IT Director I:\VB Request Memos\2014 Storage Discussion.docx