| 引用本文: | 王德文,肖 磊,肖 凯.智能变电站海量在线监测数据处理方法[J].电力自动化设备,2013,33(8): |
| WANG Dewen,XIAO Lei,XIAO Kai.Processing of massive online mornitoring data in smart substation[J].Electric Power Automation Equipment,2013,33(8): |
|
| 摘要: |
| 通过对智能变电站状态监测数据的特点以及Hadoop云计算技术的分析，研究数据检测机制以及索引构建方法，提出一种基于Hadoop的智能变电站在线监测数据可靠存储及快速查询方法。该方法将变电站在线监测所收集的海量变电设备在线监测数据冗余存储在Hadoop分布式文件系统（HDFS）中，对在线监测数据的索引表结构进行优化，并存储在分布式结构化数据库（HBase）中，实现海量在线监测数据的快速查询。搭建基于Hadoop的在线监测数据处理实验平台，进行了基准测试、排序测试以及在线监测数据读写性能测试等。测试结果表明，上述方法及平台在查询时间、读写效率及吞吐量等方面具有良好的性能，能够满足智能变电站大规模在线监测数据的可靠存储及高效处理的需要。 |
| 关键词: 智能变电站 监测 云计算 Hadoop 快速查询 数据处理 |
| DOI： |
| 分类号: |
| 基金项目:国家自然科学基金资助项目（61074078）；中央高校基本科研业务费专项资金资助项目（12MS113） |
|
| Processing of massive online mornitoring data in smart substation |
|
WANG Dewen, XIAO Lei, XIAO Kai
|
|
Department of Computer Science，North China Electric Power University，Baoding 071003，China
|
| Abstract: |
| The characteristics of smart substation monitoring data and the technology of Hadoop cloud computing are analyzed and the mechanism of data examination and the method of index establishment are studied. A method based on Hadoop is proposed for the reliable storage and fast query of smart substation online monitoring data，which stores the collected massive online monitoring data of substation equipments in ＨＤＦＳ（Hadoop Distributed File System） redundantly，optimizes the structure of index table for online monitoring data and stores it in HBase（Hadoop column-oriented dataBases） to achieve the fast query of massive online monitoring data. An online monitoring platform based on Hadoop is built，with which，the benchmark test，sort test and read／write efficiency test are carried out. Test results indicate that，the proposed method and platform have high performance in query speed，read／write efficiency，latency and throughput，meeting the requirements of smart substation for reliable storage and efficient processing of massive online monitoring data. |
| Key words: smart substation monitoring cloud computing Hadoop fast query data processing |
