Evaluating Online School Success

How will we know if the program we create is effectively providing the students with what they need? There are so many options available, either in-house and on the web, to evaluate quickly how people feel about their experience in an online school. Metrics, when used effectively, are invaluable in this regard. For us, since this is so new and so unknown, finding out first who is interested with both teachers and students, then how we have hit benchmarks of progress will be important parts of the process.

The graph at the top of the page, with information taken from the US Census Bureau, shows the dropout rate among high school students over the last 30 years. I have to think that using online classes to either keep these students in school, or to give them access to credit recovery would push us, or anyone to create these types of schools.

Survey tools to gather data

• use them to analyze where the data is telling the school to move.

65% Survey

• when a student hits the 65% completion point, they are automatically taken to a survey that asks them about teacher effectiveness and class effectiveness
• Data is then analyzed and changes are suggested for the course or teacher
• This is kind of like RateMyTeachers on steroids.

End of year, mass surveys to stakeholder groups

• each district served receives the survey
• they are asked how well the school is serving the community/school

Principals and Guidance counselors receive surveys

• they are asked how well the school is serving the community/school
• how does the virtual school compare/stack-up to the regular school

Parent Survey

• targeted to their child's schedule and class record

Student Survey

• end of course survey about their school experience

Marketing and Publicity in Virtual Schools

Establishing a virtual school requires buy in from all of the stakeholders in your community. In order to win that, and it is something that has to be won, your message has to be clear, ethical, and seen as driven from an altruistic motive, which of course it will be. This type of school is by its nature non-traditional, and will be seen as a change to current environment. It's safe to say that when you introduce change, it has to be done in a careful way, and with great care given to bringing the right people on board with you.

Who are those people? That depends on your community. With us, it begins and ends with students and their needs. A good point was brought up this morning by Dan while we were in a conversation with an administrator from Texas: "If we are preparing our students for college, this might be something we should look into because this is what they will see when they get there."

Otherwise, counselors are also people who need to brought on (see below), the public, administrators, teachers, parents, etc., are going to be wary of adding this to your district's offerings.


Marketing and Student Outreach

Sharing what courses looked like

* teachers and administrators need to see that what you are offering online is comparable to what is being offered in the classroom
* Samples from existing virtual schools

Access

* making sure that your students/teacher have access to the classes.
* Assess what your population has in terms of connection and bandwidth
o survey
o assessment of what we can offer if there is low bandwidth

Counselors are the gateway to the students getting access to the classes.

* Sell the counselors on the idea, and the students will follow into the classes.
* This makes perfect sense because counselors tend to know the needs of specific students as well as the general trends for overall grade levels because they analyze the trends that students exhibit while registering for classes.
o also can read emotional needs much more clearly


Get the name out there

* products with emblems
* How do we publicize this? And in publicizing the virtual school, would be we be unethical in relation to the other subjects? Seriously, no one advertises their Economics 101 class during scheduling?
* But why not? This might show passion and energy that student might pick up on. Interesting


Customer Service training is integral to building a successful school in this light

* treating people with care and genuine concern takes the stigma out of online classes being impersonal
* the initial meeting is the beginning of this process. Once the students are enrolled and participating in the classes, the personal nature of the teacher/student relationship furthers this.

Curriculum Design in Virtual Schools

Yesterday, most of the questions I was generating were in regard to how to actually create the content for the classes offered. Today's session will deal mainly with curriculum design and instructional strategies. What I am looking for is what type of experience students are having in this environment. When a student decides to take an online course, what will they be doing with other students? Will they collaborate, and if so, how? When I think about 21st Century skills, and creating students capable of recognizing patterns and being empathetic, I wonder about the sometimes isolating nature of the online world.


Course Development

* every course has a motif
o engagement is begun here.
o the example being given here is Comprehensive Math for 6th grade where the class is built around a zoo motif. The student is assuming the role of the zookeeper, and will use the math skills he or she learns to successfully maintain a zoo.
* When students remain the center of the class, success is inevitable
o students benefit, teachers benefit, school benefits.
* Courses are developed using a multiple intelligence modality. Learning styles are targeted through varied course content and varied choices for student output.
o allows students to feel success regardless of their talents.
* Students can show mastery in a variety of ways

Quote from the main course screen: "instructors have access to all areas of the student account. Your email account is for school use only."
Science might be a problem--how do you do labs?
Look and Feel

* material in the course, the actual pages, are chunked for the students, so that there is limited scrolling.
* courses mirror the diverse population.
* Readability levels
o chunks of contents
o web links
o other resources
o average
o courses do not use materials that dip too far below grade level, or too far above grade level
* Writing
o personal reflection
o Upper level Bloom's
o Comparable to lesson
* These two components appeal to both ends of the spectrum.

Some notes from how the course looks as you enter it:

* The first thing I noticed is that the interface is really basic, with not a lot of busy-ness on the screen.
* Also, the students must decide at what pace they will proceed. When they decide this, they must write the instructor with a rationale for their pace choice. Accelerated is 18 weeks, Traditional is 34 weeks, or Extended, where the pace is decided on by the student and teacher together.
* The autograde function is interesting in that you can design assessments to include questions that are graded by the LMS, but also have on that same assessment an essay or short-answer portion that is sent to the teacher for grading.
* What these course will always need is rich web content and collaboration. If the students could also create course content that other students use, that adds another huge element to this.
* Could we design a course around an MMORG? I am thinking of Civilizations here, where the students are required to use the game as an integral part of learning ancient civilizations, in addition to other content from a more traditional sense.

Standards and alignment

* must align to state, national and now world standards
* specific to each state
* if designed well, courses and lessons will blend the standards together seamlessly and fluidly
o standards--objective--lesson--assessment

Interactivity, Multi-Media, Technology

The nature of an online school lends itself to using multi-media as a means to create content, whether teacher-created or student-created. The rise of things like Second Life where several members of the class could meet in a quasi-physical way adds another level of meaning to this.

* How to allow them to use Web 2.0 technologies
* Can we incorporate these into our classes online?
o necessary and obvious

Assessment

* autograde- students submit a quiz online that is graded immediately
* written
* oral assessment
* project
* self-assessment

What are some of the collaborative activities that are included in the assessments?

* classmates- interaction through discussion boards
o students are given an option sometimes through the assessment piece--individual or collaborative
o this is in beta right now.
o students have the ability to reach outside of the classroom and connect
* family
* community- students can work with their communities for school projects

21st Century Skills

* Life skills- how to collaborate and how to interact
* core subjects
* learning and thinking skills
* internet and medial literacy- how to detect credibility online and how to assess what constitutes a good source.
* 21st Century Content

Resources

* web links
* guided web activities
* subscriptions
* software

Virtual School Instructional Leadership

How are teachers evaluated in an online model? Virtual schools present different challenges to administrators as they do not exist within the Hunter-esque model of evaluation. Barbara referred to this when she posted on her blog about the unique problems associated with School 2.0 and evaluations in the traditional sense.

How are teachers hired?

• teachers must first take an online survey before they are asked to come in for an interview or a phone call.
• minimum score required
• Instructional Leaders are like "principals" of the the virtual school.
• The interview process is unique
• face-to-face and phone all at once
• integral because of the nature of the teaching--a lot of the teaching is done by phone.
• There has to be some thought put into how different the hiring process will be than the traditional interview and screening process. In addition to the teaching craft, the interpersonal skills are essential to the success of the online teacher.
• Mentoring program
  
• reduced class load for the mentoring teacher
• follow-up calls and once a week to the new teacher
• The drop aspect of this is an often-discussed

Drop

Florida has a policy of open enrollment where students can drop or add the class at any point during its progression. This means that the learning and teaching is completely asynchronous on two levels:

• learners are not always accessing the information at the same time, even if they are at the same point in the class time line.
• learners in the class may be at several different points in the time line. (imagine an American History class where one group of students is working through the Depression while another group may still be working through Lincoln's presidency. The teacher is responsible for teaching, re-teaching, re-teaching, until the class has finished).

In addition to the open enrollment, teachers can also drop a student after a series of missteps or lack of performance. WHAT! Imagine that in a regular classroom!

The appearance of an online teacher, to their families, is very different.

• because the teacher works from home, families tend to view the teacher as a non-teacher. After all, they don't look like they are working because they are sitting in front of the computer or talking on the phone.
• Here is the beginning of the difference for teaching and evaluating, perhaps. When becoming an online teacher, the applicant needs to be made aware of exactly how different the profession is in this new model.
• In addition, the strategies teachers use to admonish or to motivate, can be extremely creative
• contests

How do we know that the students are the ones doing the work?

• Mirrors traditional classrooms in that the teacher will get to know the student's writing and style
• Phone conversations also let the teacher know more about the students.
• Turnitin.com
• Database of student work to compare to. As more assignments get put in, the database grows.
• Teachers can view how long a student has spent on an assignment through the LMS.
• Oral assessments
• "how did you come to that conclusion?"
• Requiring a student to take a face-to-face exam
• Students are also allowed to resubmit assignments at any point throughout the course.
• Pacing
• three paces allowed by FLVS
• traditional pace- similar to what a traditional school would offer for a full-year class (about 8mos.)
• extended pace
• accelerated pace
• flexibility within each of these paces.
• Students and parents also complete an evaluation when the student is 65% completed with the course. This feedback is used to reshape how the teachers are selected, how the classes are structured, and the type of professional development the teacher needs to focus on.
  

How do administrators know that the teachers are doing their job?

• Each teacher has credit goal that they are expected to meet.
• it becomes evident when a teacher has a weak point in their pedagogy
• There is data that may reflect
• Each teacher is required to complete so many students throughout the course of a year.
• performance-based
• The Instructional Leaders look at the teachers each week
• data from calls
• student ebb and flow from the class
• weekly student progress- pace and academic average.
• This might not reflect everyone's model, but since FLVS is performance-based, what happens when a teacher is floundering?
• when the IL picks up trends within the teacher's performance over time, then they intervene much as they would in a normal school.
  
• providing strategies and support.
• Content-buddies (at FLVS): new teachers are placed with a content-buddy as well as a mentor to help troubleshoot areas of curriculum.
• Team teaching: working with a group of teachers on a class so that they can all interact with the group of students.
• Three evaluations per teacher
• beginning of the year
• credit goal- there is a standard that all teachers are expected to meet with how many student credit-hours are completed by their classes.
  
• leadership goal- how a teacher communicates within the course and how they work within the team
  
• business goal
• professional development goal
  

Teamwork and getting teacher stakeholder buy-in is essential, and in thinking of how to set this up in my district, the decision to include some and not others will make or break whether or not the Sparta Virtual School becomes a reality. This, like many other areas of School 2.0, requires a certain ability within the stakeholders to be able to suspend disbelief and be comfortable with not knowing how it is going to look upon completion.

This is such a different model of learning, and so much of it lives in opposition to traditional teaching (hours of operation especially) that not to spend time on the training of staff would be disastrous to the success of the virtual school.

Virtual Leadership Training

Here are my notes from the Virtual Leadership Conference, Day 1. I am
just going to let these fly out there, and it may be in several parts,
but I am not usually one who writes in this format (notes, that is).

Goals of the group:

* How to build an online school
* How to grow a school wisely
* How does teaching look in this environment
* How are students successful in this environment
* How does staffing look in regards to the union issue?
* LMS’s.
* How do we monitor students and teachers
* Assessment of students? How is authentic?
* Pacing of students as they move through a course
* Funding of your own students/funding of students from outside of the district?

There is a lot of demand and a lot of need out there

* state requirements
* credit recovery
* Kids don't have issue with the online learning, we do.

How do you grow your online school and maintain quality

* million dollar question
* constant battle to maintain that balance

Florida Virtual School is funded based on student success

* at their own request
* measures in place to maintain academic integrity

What is student success in an online school? Is there a way to track student performance over time?

* SAT, ACT prep courses and affect on scores
* Do online schools show success in this area?
* Is the very nature of an online learning environment contrary to standardized testing?
o should these students be measured in another way?

Blogger is giving me issues. Here are some more ideas:

Class discussions via YackPack.

* Whatever LMS we choose, it should have the ability to me module or node-based, so we can add things like YackPack or Skype directly into them.
* elluminate is another that FLVS uses.

College Reach-out Program

* priority funding with FLVS and priority access to students in this group who are at-risk or designated for this program.
* This type of program provides students who might have slipped through the cracks or missed out on a traditional school "track" on getting themselves in position to reach the college level.
* Underserved students are viewed as not a good fit for online learning
o not true-- a populare misconception
o FLVS has data to show that students in minority populations do very well on standardized tests after having been associated with FLVS.
o Stereotype eraser, and students who might otherwise shy away from academic success, or are shunned due to social stereotypes (behavior problems, bad decisions, social stigmas) do very well in an online class.
* The idea that students are "embarrassed" to be smart in some schools due to social pressures, can be "invisibly smart" in an online class while still maintaining their social status (the sagging pants phenomena)
* Migrant students benefit from FLVS through Dan Bolton--Angels Helping Hands who provides inexpensive laptop computers to them so they can participate in online learning.

The prevailing opinion is that online learning is for the bright kids. It also attracts students who are looking for a free ride.

* 29-day grace period for students to drop the class. This usually weeds out those looking for a free-ride.
* This system also represents under-achieving students.

Initially, the focus of online learning was on the higher-achieving students, but the trend is to move away from this and to "serve the underserved."

Internet connection speed also plays a part in the creation of course content

* providing equal access for all members of the class to materials is essential to the students feeling of success and belonging in the environment.
* This could be meted out in the beginning of the class or at initial enrollment through a survey that informs the school how each student connects to the internet.
* Funding options for students with limited access to technology need to be explored.

Student support

* mentoring programs that involve community support or corporate support.
o many companies require that employees mentor students
* The need for students to rise to meet a level of success mandated by an online school, rather than being held back by it is crucial, and providing support measures, either through mentoring or through built-in measures.

Publicity and Marketing also play a huge role in the success of your online program if you plan to open it to those outside of your district. Even within district, buy-in from counselors who trust the program and push it in their meetings with students will make or break the school.


As for how to "teach" so far the best quote is "You will not work less, you will just work differently."

In addition, a rather depressing statistic for me, the Northeast is the least progressive in the area of online school proliferation when compared to the rest of the nation.

* is this union related?
* If so, this will require a lot of relationship-building with unions and with other stakeholders.
* The idea that this will change the working environment is unquestioned, but will it impact negatively?
o what will the hours of a virtual teacher look like?
o how will that be worked out in a teacher contract?