對於大量天文資料之分散式小行星軌跡探索

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：47

、訪客IP：3.135.184.109

姓名

黃柏軒(Po-Hsuan Huang) 查詢紙本館藏

畢業系所

資訊工程學系

論文名稱

對於大量天文資料之分散式小行星軌跡探索
(Distributed Asteroid Track Discovery for Large Astronomical Data)

相關論文

★ 應用自組織映射圖網路及倒傳遞網路於探勘通信資料庫之潛在用戶	★ 基於社群網路特徵之企業電子郵件分類
★ 行動網路用戶時序行為分析	★ 社群網路中多階層影響力傳播探勘之研究
★ 以點對點技術為基礎之整合性資訊管理及分析系統	★ 在分散式雲端平台上對不同巨量天文應用之資料區域性適用策略研究
★ 應用資料倉儲技術探索點對點網路環境知識之研究	★ 從交易資料庫中以自我推導方式探勘具有多層次FP-tree
★ 建構儲存體容量被動遷徙政策於生命週期管理系統之研究	★ 應用服務探勘於發現複合服務之研究
★ 利用權重字尾樹中頻繁事件序改善入侵偵測系統	★ 有效率的處理在資料倉儲上連續的聚合查詢
★ 入侵偵測系統：使用以函數為基礎的系統呼叫序列	★ 有效率的在資料方體上進行多維度及多層次的關聯規則探勘
★ 在網路學習上的社群關聯及權重之課程建議	★ 在社群網路服務中找出不活躍的使用者

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

[檢視]

[下載]

本電子論文使用權限為同意立即開放。
已達開放權限電子全文僅授權使用者為學術研究之目的，進行個人非營利性質之檢索、閱讀、列印。
請遵守中華民國著作權法之相關規定，切勿任意重製、散佈、改作、轉貼、播送，以免觸法。

摘要(中)

小行星(asteroid)遍布於整個太陽系當中，受到重力吸引而造成其移動的現象(moving object)。隨著硬體設備技術的進步，天文資料的獲取越來越快速。於大量的望遠鏡觀測資料當中尋找出小行星軌跡，將可以幫助天文學家了解更深層的天文意義，以及對於軌跡的路徑是否對地球造成危害作更進一步的預測。然而天文觀測資料的龐大，透過手動或人工的方式計算軌跡過於費時費力，且精確性較低。
為了要減少硬碟的寫入寫出，解決記憶體不足的問題，本論文將導入處理大量資料的方法，採用分散式系統，運用分散式的演算法，以具更系統性的方法尋找小行星軌跡。本論文將使用分散式的檔案系統與資料庫來做為儲存的設備，以良好可靠性與擴充性作為考量。同時應用分散式的Hough Transform演算法，搭配Open Source的雲端計算環境，更有效率地尋找小行星軌跡。
　　在實驗的部分，本論文將於Palomar Transient Factory(PTF)天文觀測資料中找出小行星軌跡。本論文的方法明顯的讓軌跡探索的效率有所改進，對於未來新獲取的觀測資料也能進行有效率的更新，並提供視覺化的介面來觀察小行星軌跡的移動。

摘要(英)

Asteroids are spread in the whole Solar System. They are attracted by gravity which makes them a moving object. As the hardware device improve, acquiring astronomical data becomes faster. Discovering asteroid track in the large observed data can benefit astronomical researchers in understanding some astronomical phenomena and can be examined for further prediction to whether it causes disasters to Earth. Due to large astronomical data, finding out asteroid track by manual is absolutely hard, inefficient and low accuracy.
In order to reduce disk I/O overhead and solve the problem of memory insufficient. This research proposes to adopt a distributed Hough Transform method with Big-Data data management and cloud computing technique for systematically searching asteroid tracks. This research utilizes distributed file system and database as the storage based on good reliability and scalability. It also uses an open source cloud computing environment for more efficiency in searching asteroid tracks.
　　In the experiments, this research discovers asteroid tracks in Palomar Transient Factory astronomical observed data. The results represent that our method can improve the efficiency apparently. New observed data can be also updated efficiently and provide the interface of visualization for observing the moving of asteroid.

關鍵字(中)

★ 大量資料
★ 雲端運算
★ 小行星軌跡
★ Hough Transform演算法

關鍵字(英)

論文目次

摘要 i
Abstract ii
誌謝 iii
目錄 iv
圖目錄 vi
一、緒論 1
1-1 研究背景 1
1-2 研究動機與目的 2
1-3 論文架構 2
二、文獻探討 4
2-1 巡天計畫 4
2-2 小行星 5
2-3 MapReduce架構 5
2-4 Hadoop 6
2-5 OpenStack 7
2-6 小行星軌跡探索 8
三、系統架構 9
3-1 雲端運算平台 9
3-2 HDFS檔案系統 10
3-3 HBase資料庫 11
3-4 前處理階段(Preprocessing) 14
3-5 Hough Transform階段(HT Stage) 16
3-6 Track Discovery階段(TD Stage) 16
四、研究方法 17
4-1 Hough Transform演算法 17
4-1-1 分散式Hough Transform演算法 20
4-1-2 天文資料的分割 21
4-1-3 HT Stage的MapReduce 23
4-2 小行星軌跡的延伸 24
4-2-1 小行星軌跡的更新 26
4-2-2 TD Stage的MapReduce 30
4-3 視覺化查詢界面 32
五、實驗 36
5-1 HT Stage執行時間 37
5-1-1 HT Stage基於HDFS 37
5-1-2 HT Stage基於HBase 41
5-2 TD Stage執行時間 49
5-2-1 小行星軌跡延伸 49
5-2-2 小行星軌跡更新 51
六、結論 54
參考文獻 55

參考文獻

[1] R. Duda and P. Hart, “Use of the Hough Transformation to Detect Lines and Curves in Pictures”, Communications of the ACM CACM Homepage archive Volume 15 Issue 1, Pages 11-15, Jan. 1972.
[2] Apache Hadoop, http://hadoop.apache.org/
[3] Apache HBase, http://hbase.apache.org/
[4] OpenStack, https://www.openstack.org/
[5] C. Zhai, M. Shao, B. Nemati, T. Werne, H. Zhou, S. G. Turyshev, J. Sandhu, G. Hallinan, and L. K. Harding, “Detection of A Faint Fast-moving Near-earth Asteroid Using The Synthetic Tracking Technique”, The Astrophysical Journal, 792:60 (14pp), 2014 September 1.
[6] Random Thoughts on Coding, http://codingjunkie.net/secondary-sort/
[7] P. Gural, J. Larsen, and A. Gleason, “Matched filter processing for asteroid detection”, Astronomical Journal vol. 130 pp. 1951-1960 2005.
[8] Hadoop 101: Programming MapReduce with Native Libraries, Hive, Pig, and Cascading, http://blog.pivotal.io/pivotal/products/hadoop-101-programming-mapreduce-with-native-libraries-hive-pig-and-cascading
[9] Palomar Transient Factory, http://www.ptf.caltech.edu/
[10] B. Shucker and J. Stuart, “Detecting asteroids with a multi-hypothesis velocity matched filter”, Proc. Asteroids, Comets, and Meteors, 2008.
[11] D. Han and E. Stroulia, “A Three-Dimensional Data Model in HBase for Large Time-Series Dataset Analysis”, MESOCA, pp. 47-56, 2012.
[12] Coding Hassle, http://codinghassle.blogspot.tw/2015/03/hadoop-shuffle-and-sort-phase.html
[13] S. Daruru, S. Dhandapani, G. Gupta, I. Iliev, W. Xu, P. Navratil, N. Marin,J. Ghosh, “Distributed, Scalable Clustering for Detecting Halos in Terascale Astronomy Datasets”, Data Mining Workshops (ICDMW), pp. 138-147, 2010.
[14] B. MA, A. Shoshani, A. Sim, K. Wu, Y. Byun, J. Hahm, M.-S. Shin, “Efficient Attribute-based Data Access in Astronomy Analysis”, High Performance Computing, Networking, Storage and Analysis (SCC), pp. 562-571, 2012.
[15] D. Carstoiu, E. Lepadatu, M. Gaspar, “HBase - non SQL Database, Performances Evaluation”, International Journal of Advancements in Computing Technology
Volume 2, Number 5, December 2010.
[16] G. Mackey, S. Sehrish, J. Bent, J. Lopez, S. Habib, J. Wang, “Introducing Map-Reduce to High End Computing”, Petascale Data Storage Workshop, pp. 1-6, Nov. 2008.
[17] Y. Liu, B. Chen, W. He,Y. Fang, “Massive Image Data Management using HBase and MapReduce”, Geoinformatics (GEOINFORMATICS), pp. 1-5, 2013.
[18] Y. Chen, W. Li, J. Li, and T. Wang, “NOVEL PARALLEL HOUGH TRANSFORM ON MULTI-CORE PROCESSORS”, Acoustics, Speech and Signal Processing, 2008. ICASSP, pp. 1457-1460, 2008.
[19] R. K. Satzoda, S. Suchitra, and T. Srikanthan, “Parallelizing the Hough Transform Computation”, IEEE SIGNAL PROCESSING LETTERS, VOL. 15, pp. 297-300, 2008.
[20] L. Denneau, R. Jedicke, T. Grav, et al., “The Pan-STARRS Moving Object Processing System”, PASP 125, pp. 357–395, 2013.
[21] M. Shao, B. Nemati, C. Zhai, S. G. Turyshev, J. Sandhu, G. W. Hallinan, L. K. Harding, “Finding Very Small Near-Earth Asteroids using Synthetic Tracking”, The Astrophysical Journal, 782:1 (10pp), 2014 February 10.
[22] L. Cai, S. Huang, “Performance analysis and testing of HBase based on its architecture”, Computer and Information Science (ICIS), pp. 353-358, 2013.
[23] V. Krishna Konishetty, K. Arun Kumar, K. Voruganti, G. V. Prabhakara Rao, “Implementation and evaluation of scalable data structure over HBase”, International Conference on Advances in Computing, Communications and Informatics (ICACCI-2012), pp. 1010-1018, 2012.
[24] J. Bhogal, I. Choksi, “Handling Big Data using NoSQL”, Advanced Information Networking and Applications Workshops (WAINA), pp. 393-398, 2015.
[25] S. Ibrahim, H. Jin, L. Lu, L. Qi, S. Wu, X. Shi, “Evaluating MapReduce on Virtual Machines: The Hadoop Case”, Cloud Computing Lecture Notes in Computer Science Volume 5931, pp 519-528, 2009.
[26] S. Moon, J. Lee, Y. Suk Kee, “Introducing SSDs to the Hadoop MapReduce Framework”, Cloud Computing (CLOUD), pp. 272-279, 2014.
[27] S. S. Sathyanarayana, R. K. Satzoda, and T. Srikanthan, “Exploiting Inherent Parallelisms for Accelerating Linear Hough Transform”, Image Processing, IEEE Transactions on Volume:18 Issue: 10, pp. 2255-2264, 2009.
[28] The Khangaonkar Report, http://khangaonkar.blogspot.tw/2013/04/using-hbase-part-2-architecture.html
[29] Big data, http://hadoopbigdatas.blogspot.tw/2013/03/hbase-architecture.html
[30] Sachin Puttur: Big Data: Overview of apache Hadoop, http://www.sachinpbuzz.com/2014/01/big-data-overview-of-apache-hadoop.html
[31] Hough Transform, http://en.wikipedia.org/wiki/Hough_transform
[32] Chair for Computer Aided Medical Procedures & Augmented Reality Lehrstuhl für Informatikanwendungen in der Medizin & Augmented Reality, http://campar.in.tum.de/Students/DaPentenrieder
[33] J. Dean and S. Ghemawat, “MapReduce: Simplified Data Processing on Large Clusters”, OSDI′04: Sixth Symposium on Operating System Design and Implementation,
San Francisco, CA, December, 2004.
[34] 小行星, https://zh.wikipedia.org/wiki/%E5%B0%8F%E8%A1%8C%E6%98%9F

指導教授

蔡孟峰(Meng-Feng Tsai)

審核日期

2015-7-28

推文