Abstrakti
We have used log file analysis in mining expected
behavior in intelligent transportation systems involving spatial
and temporal data. The challenge is how to extract complex
behavior from multiple traces, in which linear log analysis
proceeding in a row by row order does not suffice. Complex
Event Processing (CEP) is close to our need, but it is surprisingly
difficult to set up and deploy general purpose frameworks to the
purpose. This paper originates from the need to compare our
custom LOGDIG tool to Apache Flink. This paper focuses on the
deployment effort of the two, for which reason we consider
setting up the development and run-time environments, selecting
the proper analysis approach and evaluating the difficulty in five
different aspects. While LOGDIG is written solely in Python,
Flink is a combination of many languages, libraries, packages
and tools. Our comparison includes Flink in batch and stream
processing modes using external and internal preprocessing. We
lend the Degree of Difficulty (DoD) measure from sports to assess
the deployment effort. Flink needs significant setup effort for
deploying the same functionality as LOGDIG. The former is
continuously developing while LOGDIG is more focused and
stable and can be used more easily off-the-self.
behavior in intelligent transportation systems involving spatial
and temporal data. The challenge is how to extract complex
behavior from multiple traces, in which linear log analysis
proceeding in a row by row order does not suffice. Complex
Event Processing (CEP) is close to our need, but it is surprisingly
difficult to set up and deploy general purpose frameworks to the
purpose. This paper originates from the need to compare our
custom LOGDIG tool to Apache Flink. This paper focuses on the
deployment effort of the two, for which reason we consider
setting up the development and run-time environments, selecting
the proper analysis approach and evaluating the difficulty in five
different aspects. While LOGDIG is written solely in Python,
Flink is a combination of many languages, libraries, packages
and tools. Our comparison includes Flink in batch and stream
processing modes using external and internal preprocessing. We
lend the Degree of Difficulty (DoD) measure from sports to assess
the deployment effort. Flink needs significant setup effort for
deploying the same functionality as LOGDIG. The former is
continuously developing while LOGDIG is more focused and
stable and can be used more easily off-the-self.
| Alkuperäiskieli | Englanti |
|---|---|
| Otsikko | 3RD IEEE International Conference on Industrial Cyber-Physical Systems |
| Kustantaja | IEEE |
| Sivut | 137-142 |
| Sivumäärä | 6 |
| ISBN (painettu) | 978-1-7281-6389-5 |
| DOI - pysyväislinkit | |
| Tila | Julkaistu - 12 kesäk. 2020 |
| OKM-julkaisutyyppi | A4 Artikkeli konferenssijulkaisussa |
| Tapahtuma | IEEE International Conference on Industrial Cyber-Physical Systems - Tampere, Suomi Kesto: 10 kesäk. 2020 → 12 kesäk. 2020 |
Conference
| Conference | IEEE International Conference on Industrial Cyber-Physical Systems |
|---|---|
| Maa/Alue | Suomi |
| Kaupunki | Tampere |
| Ajanjakso | 10/06/20 → 12/06/20 |
Julkaisufoorumi-taso
- Jufo-taso 1