Echolocation

In echolocation, a sound signal is generated and transmitted. The sound signal reflects off objects in the operating environment and returns, and the round-trip time taken is used to estimate the distance to these objects (see Figure 3). Echolocation is used for ranging by a wide variety of animals including bats and dolphins.

In what follows, a simple mathematical model of echolocation is presented. Let the transmitted signal be given by $x\left(t\right)$ . After reflection and propagation, the received signal, $y\left(t\right)$ , can be modeled as a scaled and noisy version of the transmitted signal, $x\left(t\right)$ , i.e.,

where the scaling factor, $\alpha$ , is modeled as a random variable whose distribution is dependent on the environment 32, the additive noise, $n\left(t\right)$ , is modeled as a Gaussian random variable, independent of $\alpha$ , and the time delay, $t_0$ , is the round-trip time of the signal.

At the receiver, the correlation between the transmitted signal and the received signal is calculated as

The cross-correlation signal, $r_{xy}\left(\tau \right)$ , has a peak at $\tau =t_0$ , indicating that the round-trip time to the object is $t_0$ . The distance to the object can then be calculated as

where $c$ is the velocity of propagation of sound.

Using the Reflections app

The Reflections app implements echolocation on an Android platform. Using a dialog (see Figure 4 ), a user-defined signal is configured and generated. Users can configure different signal characteristics such as signal types, signal lengths, signal shaping, and where applicable, start and end frequencies. The selectivity of the peak detection algorithm can also be selected from this interface.

This generated signal is then transmitted from the speaker of the Android device as audio, and several copies return after striking objects in the environment. Using the original signals and the returns, the cross-correlation in (Equation 2 ) is calculated. However, since the data is discrete and has a finite number of samples, the correlation is performed using a summation rather than an integral as follows:

where $\hat{x}\left(n\right)$ and $\hat{y}\left(n\right)$ are discretized versions of the transmitted and received signals, $x\left(t\right)$ and $y\left(t\right)$ , respectively.

To allow fast computation of the cross-correlation, the fast Fourier transform (FFT) is utilized 18, 32. Round-trip time and distance to the objects can be estimated using Equation 3 and Equation 4 .

These distance estimates are presented to the user as a plot on a cross-correlation graph. By pushing the interleaved arrows button, one-way travel times of the signal are shown on the cross-correlation plot. All the peaks in the cross-correlation of the return signal are identified and marked on the cross-correlation plot using a peak-detection algorithm. These peaks can be labeled either with the round-trip time in seconds, or the distance in meters to the object causing the reflection. An example of the resulting plot with a distance estimate is shown in Figure 5 , where an object is detected half a meter from the smartphone. The distance measurements are shown in Figure 5 (top) and the cross-correlation plot is labelled with one-way trip times in Figure 5 (bottom).

A cursor to provide details on points of interest on the cross-correlation signal has been implemented. Other features include pan and zoom.

Supporting Notes, Exercises, and Quizzes

One of the main components of the eModule is the embedded notes that can be accessed on the app itself or from the companion website 17. These notes provide educational content suitable for several educational levels (see Figure 6 for an example). Practice problems are provided, along with quizzes at each skill level. Each quiz, upon completion, gives the user the correct answers, as well as explanations for all answers. The quiz interface points to appropriate locations in the notes for further study.

Companion Videos

In order to better demonstrate concepts to students, videos are used. The videos are embedded in the app, and are also available on YouTube and the companion website 17. The videos serve two purposes:

• Description of concepts including publicly available videos and lecture videos that complement the app

• Demonstrations of how the app can be used for teaching specific lessons, and serve as an aid to teachers to help guide their lesson plans (see Figure 7 ).

At the Undergraduate Level

Workshops were conducted in a junior-level signals and systems class, and in a senior level signal processing class to evaluate the effectiveness of the app in demonstrating concepts in signal processing. These workshops were scheduled to be ninety minutes each. This time included administering a pre-quiz, providing the students a brief overview of the activities, time for hands-on activities, and the post-quiz. Overall, the students had over an hour for the hands-on activities, and found that to be sufficient to carry out all workshop activities.

Demonstrating Signal Processing Concepts

A workshop was conducted at Clarkson University’s junior-level Systems and Signal Processing class. A total of 67 students participated in the workshops. The benefits of using the app for teaching concepts was measured using pre/post quizzes. Specific concepts covered were:

• Autocorrelation;

• Round-trip time;

• Distance Estimate;

• Correlation;

• Symmetry of the autocorrelation; and

• Noise effects.

After the pre-quiz, a short lecture was presented to the students describing how to use the app. Students were then asked to perform a series of hands-on activities. These allowed the students to, in sequence, familiarize themselves with the app, verify the effectiveness of ranging using the app, and connect the steps in echolocation to abstract concepts covered in class prior to that point. Additionally, students also experimented with changing signal types, signal duration, signal shapes, and other signal parameters.

After the hands-on exercises, in order to assess the effectiveness of the workshop and the Reflections app, a post-quiz was administered to the students. Finally, as survey was given in order to obtain student experience with the app.

By looking at the results, it is clear that (see Figure 8 ) students are able to learn concepts including what signals help with better correlation results; the role of correlation in ranging; and several other areas of interest, such as the effect of noise on signal processing algorithms. A statistical analysis was performed on the results. Using a two sample t-test 33 , we obtained a p-value of $6.1 \times {10}^{–5}$ , indicating that the null hypothesis was rejected, and confirming the alternative hypothesis that students did show improvement. Furthermore, as shown in Figure 9, most students agreed that “The app was fun/easy to use” and “The workshop was helpful.”

Demonstrating Machine Learning Concepts

With the increasing impact of machine learning and artificial intelligence (AI) on society 34, and the increased awareness of these topics among students 35, there is a growing need to teach these concepts at the undergraduate level. However, the methods used to teach these concepts need to be appropriate to the level of understanding of the students.

To accomplish this task, we use one of the applications of the Reflections app, which is learn how to make the best use of a given space36 by characterizing its acoustic properties. Since the Reflections app reacts to the objects in the environment, the app is ideally suited for the space characterization task.

Audio cross-correlation data obtained from the app can be fed into a machine learning algorithm that then determines the best use of the space. Concepts that can be reinforced include machine learning in signal processing, feature identification and extraction, and decision making in artificial intelligence (AI).

A workshop was conducted at Clarkson University. Students from two-classes participated: a senior-level Digital Signal Processing class and a senior-level Signal Processing and Applications course at Clarkson University, specifically designed for students not in the Electrical Engineering major 37. Students in this course included a mix of Computer Engineering majors, Software Engineering majors, and Mechanical Engineering majors. The total number of students who participated in the workshops was 32.

A pre-quiz covering the basics of machine learning was administered, followed by a brief introduction to concepts, as well as hands-on activities. The hands-on activities included room classification using data collected from the Reflections app.

Audio signals from the Reflections app were transmitted into the center of a room, as well as from the center of a room as shown in Figure 10 . The acoustical profiles of each room were estimated using the Reflections app and collecting cross-correlation data from different points in the room.

With this data, students were given two tasks: feature selection and classification. For classification, students were introduced to several algorithms including support vector machines (SVM), k-nearest neighbors (kNN), and random forests. Students were asked to investigate two types of feature selection: intuitive features such as peak distances, the distance between peaks, etc.; and features extracted using energy methods such as principal component analysis (PCA). Using a combination of extracted features, students were asked to evaluate each classification algorithm for room classification.

In a post-quiz and survey, the students were retested on the same topics as the pre-quiz and were asked questions to determine if the students found the hands-on activities helpful in understanding machine learning concepts. Specific topics covered in each question that was asked in the pre-/post-quiz are:

• Comparison between kNN and SVM

• Properties of a kernel SVM classifier

• Cross-validation

• Supervised machine learning

• Random forests

• Principal component analysis (PCA

The survey question asked was “This workshop was helpful and informative.” The answer was collected using a one to five Likert-type scale, where 1 indicated “Not at all helpful”, and 5 indicated “Very helpful”.

The results of the pre/post quiz are shown in Figure 11 , and the results of the survey question are shown in Figure 12 . Students were able to show improvement in understanding of machine learning concepts as can be seen from the results of the pre/post quiz. A statistical analysis was performed on the results. Using a two sample t-test, we obtained a p-value of $0.005$ , rejecting the null hypothesis, and confirming the alternate hypothesis that students showed improvement. The students also found the workshop to be helpful in getting acquainted with machine learning concepts.

At the K-12 Level

Our focus in this work is the “science and engineering practices” module of the Next Generation Science Standards (NGSS). We design our lessons to help teachers and students apply these engineering practices in their lessons. Using this technology and building them into lesson plans help us address the science and engineering practices as identified by NGSS 38.

We use the Reflections eModule app in consultation with middle school and high school teachers. We work with teachers to develop lesson plans that align with topics already being covered in class. By using the eModule, the teachers are able to seamlessly integrate science and engineering practices into their existing lesson plans.

A workshop was held at Clarkson University in December 2018, as part of the Master Teacher “Teacher to Teacher” Conference. The participants were K-12 teachers who have been selected to be Master Teachers39 in both physical sciences and life sciences. Participating in the workshop were 12 New York State master teachers. In the workshop, a lesson plan was developed based on the Reflections app that covered multiple topics in physical and life sciences. The format of the lesson plan is based on the Masters of Arts in Teaching (MAT) lesson template developed at the Clarkson University Department of Education 40. The lesson plan is shown inFigure 13 .

Prior to the start of the workshop, the teachers were asked two questions:

• “How comfortable are you using technology in your classroom?” (One to five Likert-type scale)

• “Do you think using technology in your classroom will help increase student engagement?” (Yes or no question)

The responses to these questions are shown in Figure 14. We can conclude that while a significant portion of the participants was hesitant in using technology in their classrooms, they all believe that using technology in classrooms will improve student engagement.

At the end of the workshop, we asked the teachers the following questions:

• “Has this workshop improved your understanding of science and engineering practices?” (Yes or no question)

• “How willing are you to use technology in your classrooms?” (One to five Likert-type scale)

The responses to these questions are shown in Figure 15 . It is clear from these responses that being exposed to a simple methodology to incorporate science and engineering practices into lessons, with the help of the Reflections app, allows teachers to think more positively about science and engineering practices, and to include them in their lessons.

Interdisciplinary extensions

The Reflections app has been designed as an app for implementing echolocation. As such, it has been used to teach concepts in sound propagation, waveform shapes, correlation, and machine learning. By extending the applications that the app can be used for, the number of concepts that can be covered increases as well.

For example, the echolocation app can be used to distinguish between the physical characteristics of different objects based on the signature of the reflected sound. A hard object would have a different signature, compared to a soft object. Different types of material can then be distinguished from each other using a classifier. Topics that be covered using this approach include classification, feature extraction, and signature detection.

Future Directions

In this paper, we presented one way of introducing machine learning content to students: via stand-alone workshops. Another approach is by integrating content into the curriculum of a course. The effectiveness of this method will be investigated by the authors in future work. As part of these assessments, we will include a control group to better highlight the effectiveness of our approach. With regard to the K-12 aspect, we will also contact the teachers from the Master Teacher workshop in order to see if they are using the science and engineering practices discussed in the workshop. We will elicit information about the methods they are using as well as the tools they are using.

Currently, the Reflections app is a stand-alone app, and students can work together to facilitate group work. For future online classes and remote reaching applications, the users of the app need the capability to collaborate in a networked environment. One avenue of future work is to incorporate a built-in collaboration environment that enables screen-sharing, data-sharing, and a chat interface.

Possible future extensions to the project include the development of new eModules. Possible areas of interest include concepts and applications that have relevance to technology that students regularly use. Examples include localization (GPS and similar applications), speech processing (motivated by phone calls), and data communications and data protection (cybersecurity).

These extensions will allow this app and future eModules to reach a wider audience of educators and students.