You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

164 satır
5.6KB

  1. \pdfminorversion=4
  2. \documentclass{beamer}
  3. \usepackage[T1]{ fontenc }
  4. \usepackage[utf8]{ inputenc }
  5. \usepackage{ lmodern }
  6. \usepackage{ color }
  7. \usepackage{ graphicx }
  8. \graphicspath{ {./pix/} }
  9. \usepackage{ drawstack }
  10. \usetheme{metropolis}
  11. \title{Customs tests in Free Electrons's lab}
  12. \subtitle{}
  13. \date{\today}
  14. \author{Florent (Skia) Jacquet}
  15. \institute{Free Electrons}
  16. \begin{document}
  17. \maketitle
  18. \begin{frame}[fragile]\frametitle{Why custom tests}
  19. \begin{itemize}
  20. \item Just booting is not enough.
  21. \item Need to test some specific components such as USB, SATA,
  22. network...
  23. \item Also need to test with custom kernel images, or custom rootfs, or
  24. both.
  25. \item Need to be able to send jobs, on demand, easilly, on the specified
  26. boards, while being notified at the end.
  27. \end{itemize}
  28. \end{frame}
  29. \section{Overall architecture}
  30. \begin{frame}[fragile]\frametitle{Many tools}
  31. \begin{itemize}
  32. \item LAVA
  33. \item Buildroot
  34. \item The custom test tool
  35. \item The test suite
  36. \end{itemize}
  37. \end{frame}
  38. \begin{frame}[fragile]\frametitle{LAVA}
  39. \begin{itemize}
  40. \item Runs jobs on devices.
  41. \item Runs KernelCI jobs: only boot.
  42. \item Provides an API to do many thing. Among them: sending jobs and
  43. querying results.
  44. \end{itemize}
  45. \end{frame}
  46. \begin{frame}[fragile]{Buildroot}
  47. \begin{itemize}
  48. \item Used to make only rootfs.
  49. \item Is configured to provided the needed tools such as iperf,
  50. bonnie++, etc...
  51. \item Builds fresh rootfs every night for all the needed architectures:
  52. ARMv4, ARMv5, ARMv7, ARMv8.
  53. \item Stores them in a LAVA-accessible folder.
  54. \end{itemize}
  55. \end{frame}
  56. \begin{frame}[fragile]{The custom test tool}
  57. \begin{itemize}
  58. \item Has jobs templates and knows how to send them.
  59. \item Can upload custom artifacts useable for tests.
  60. \item Can ask LAVA to notify given emails addresses when the job ends.
  61. \item Needs an access to the LAVA API, an SSH access to the LAVA server
  62. to upload artifacts, and a KernelCI API token to go fetch the latest
  63. builds if no custom artifacts are provided.
  64. \item Keeps a \verb$boards.py$ file to store the boards configuration.
  65. \end{itemize}
  66. \end{frame}
  67. \begin{frame}[fragile]{The test suite}
  68. \begin{itemize}
  69. \item Is a bunch of shell scripts that will be run on the devices.
  70. \item Also needs YAML files to interface with LAVA.
  71. \item Leaves in one of our git repo on Github \footnote{\url{https://github.com/free-electrons/test_suite}}.
  72. \end{itemize}
  73. \end{frame}
  74. \begin{frame}[fragile]{Workflow}
  75. \begin{itemize}
  76. \item Daily jobs:
  77. \begin{itemize}
  78. \item A cron job runs at 23:23 to:
  79. \begin{itemize}
  80. \item Build fresh rootfs.
  81. \item Launch custom tests on every boards with there
  82. default configuration.
  83. \end{itemize}
  84. \item Another cron is run at 11:00 to get the results of the
  85. last 24 hours and send mails given the failed tests and
  86. which email address is subscribed to what board.
  87. \end{itemize}
  88. \item On demand jobs:
  89. \begin{itemize}
  90. \item Someone installs and configures the custom test tool on
  91. his/her laptop.
  92. \item He/She can send jobs, specifying whether to use custom
  93. boot artifacts from his/her laptop, or to use KernelCI's
  94. one, and in that case, which branch on which tree to use.
  95. \item He/She gets notified once the job's done.
  96. \end{itemize}
  97. \end{itemize}
  98. \end{frame}
  99. \section{How do tests works?}
  100. \begin{frame}[fragile]{LAVA v2 jobs}
  101. \begin{itemize}
  102. \item Jobs are YAML files that, combined with the device-types and
  103. devices dictionaries in the LAVA configuration, describe what to do
  104. with which device.
  105. \item It contains:
  106. \begin{itemize}
  107. \item Which device(s) to use
  108. \item The artifacts informations (URL, type, ...)
  109. \item How to boot them (Uboot, fastboot, ramdisk, nfs, shell
  110. strings to expect, ...)
  111. \item The tests to run once booted.
  112. \item Some metadata and various job informations (priority,
  113. notifications, timeouts, ...)
  114. \end{itemize}
  115. \end{itemize}
  116. \end{frame}
  117. \begin{frame}[fragile]{How do LAVA runs test?}
  118. \begin{itemize}
  119. \item Before bringing up the board, LAVA:
  120. \begin{itemize}
  121. \item Fetches the artifacts.
  122. \item Applies the needed modifications (append DTB, mkimage,
  123. modules, ...)
  124. \item Also downloads the test suite and puts it in the rootfs.
  125. This moment also makes some magic to provide some helpers
  126. useable in the test scripts.
  127. \item Finally connects itself and powers up the board.
  128. \end{itemize}
  129. \item After the board is booted:
  130. \begin{itemize}
  131. \item Runs each test described in the job definition.
  132. \item For each call to the `lava-test-case` function, creates
  133. the results objects.
  134. \end{itemize}
  135. \end{itemize}
  136. \end{frame}
  137. \begin{frame}{Questions}
  138. \begin{center}
  139. \alert{Question?}
  140. \end{center}
  141. \end{frame}
  142. \end{document}