Written tests are for the purpose of establishing a *baseline* from which airman certification is to take place. Knowledge tests demonstrate just that: knowledge alone. One must wait until the administration of the practical test so that understanding of content may be assessed via an oral examination and practical demonstration of skills. Should an applicant's knowledge — as reported by the written test — indicate a deficiency when orally quizzed, this would be instantly recognized by the DPE or FSDO Inspector.

There is some learning to be had by memorizing the answers to hundreds of possible exam questions. Especially if those questions are well-designed. So I say publish them.

The endless review for my PVT and IFR written left some useful info in my head after 10 years - mostly about regs and airspace. But the most effective and lasting learning for me was in the airplane or in the sim. Modern tools like iPad apps and simulators for context based learning would be far better for the X-Box generation. I also remember having to memorize a wrong answer so I could get it right because the FAA never got around to fixing the test. That told this aspiring pilot more than anything how the FAA worked.

In 16 years of flying, I have never flown a plane that had an ADF. I have NEVER used an NDB, wouldn't know how to use one if I had to. I have logged thousands of hours and never needed this knowledge. If I hadn't been able to just memorize the NBD questions, I would probably never passed the test. I say, publish the questions and in the meantime, drop those questions since the government is turning off NDB's anyway.

Publishing the test questions invalidates the test. Examiners consistently encounter applicants during the practical test who have acheived 100% on the knowledge test, but know little if anything about the subject matter tested. If a student is given prooper ground school training, that student will know the answers to the questions. No body's life was EVER saved by knowing that the answer to the question that begins, "The true airspeed of the airplane..." is the answer that begins, "146 knots..." If the questions are to be released, I suggest they be in the form of a study guide for the oral portion of the test. After all, a student who completes the test in 15-20 minutes has done so because he memorized answers, not because he knows the material. I doubt that we could find one accident or incident where the probable cause or contributing cause could be traced back to not having the test questions available -- a few infractions of FARa perhaps, but not an accident.

As aviation educators, we should be very concerned and interested that our students know the mnaterial and concepts covered but they should be able to demonstrate that during a thorough oral examination using scenario-based questioning.

There are several industry {learning} standards by which text questions can be validating. Giving them freely to the public is not one of those.

It would be intesting to see how many of the training and testing material organizations employ instructional designers.

The deeper question is why do we allow the government to decide these things. Flight Training should be done privately and only the insurance companies(because they stand to lose money in crashes)pilots and aircraft owners should decide who can pilot their airplanes. Government agencies are incompetent and only change the rules to appear to be doing something useful.

Too many people fail the test, start publishing the questions. Pass rates get too high, so they quit publishing the questions. This makes perfect sense to a bureaucrat who doesn't care about flight safety, just his government pay and benefits.

Believe it or not I learned and retained a lot of knowledge from my mechanic test by reading the question and learning the answer. As an A&P mechanic I do not know it all about every aircraft and it would take years to learn. The FAA has been slipping in test where there was no question or answer that are completely ridiculous and people are failing the test. Most airmen get their license and it is a license to learn and gain knowledge. If the FAA changes these exams we are going to have a real shortage of mechanics passing the written's. It should be left up to the FAA DME to determine if the student is qualified during the Oral and Practical. Things have worked so well for years, why screw it up? Del

I'm all for keeping the test questions secret as long as there is a framework to study against and practice exams to work with. I'm in the IT world and while the formal tests aren't typically public, there are TONS of practice tests out there to take. There's not guarantee that anything you get tested on in the practice exam will ever be asked during the actual exam.

I'm also with William Jones with regard to relevant questions. I'm literally right in the middle of working toward my PPL from a Sport Pilot License. The plane I own and fly doesn't have an ADF, OBS or anything to navigate with instruments beyond just the compass (which really isn't accurate - but that's a separate issue). I fly strictly with the on-board GPS. Guess how many questions there are on the PPL written exam around a GPS? Two - exactly. And neither have anything to do with actually using the GPS. I would think knowledge of how to use a GPS on any plane that has one to EXTREMELY relevant to be tested on.

Additionally, who exact does x-country flight planning on paper anymore? I haven't owned a chart since I took my checkride for my initial license - and I don't intend upon buying another one. I certainly appreciate the effort in teaching the basics - the basics are important, but they should only provide the foundation from which to teach and test against, not the rule.

So Brian, what exactly is your plan when the GPS goes blank? Hint: charts and an old E6-B can make it a non-event.

I have the on-board GPS, an iPad and an iPhone, each with their own independent batteries. If all 3 go blank at the same time I'm landing the plane and finding out why I had 3 GPS failures inside of a 2 to 3 hour flight. Option 1. I didn't get an adequate briefing from weather to find out about solar flares. Option 2. someone set off an EMP - which effectively would kill the plane as well, or Option 3. someone set off a nuclear bomb.

Your argument is the same one I constantly hear from a lot of old time pilots - many of whom don't even carry charts or E6-B's anymore.

To me, this issue and discussion reinforces the benefits of the checks and balances provided by the current written, oral, and practical test sequence. One would hope that someone that merely memorizes the written test will reveal themselves through deficiencies in the other two. In my training, I went through the entire test battery and then scored my answers using the provided answer key, following up on those I missed. It was an effective learning technique for me. Not having the answers would make it far less useful.

Having obtained both PPL and IFR within the past eight years, I agree with publishing the questions. There is considerable value in seeing what the FAA thinks is the correct answer. It is a good learning tool. Unfortunately, the test is quite out of date. Weather is no longer obtained as outlined in the test. Navigation is no longer by NDB. Only two questions pertain to GPS. And, there are several "gotcha" questions. The Kings even point these out in their courses. I still remember much of what I learned studying the test questions, even though I still can't fly an NDB approach.

For CFIs who are concerned that publishing the questions allows students to pass the test without obtaining the knowledge, I'd like to ask: didn't you have to sign the student off?

I did self-study and was ready for the ground test before I ever started flight training, because it was all I had the money for at the time. When I asked my CFI to sign me off after 3 lessons, he had me come in for a separate ground session where he grilled me for about an hour, to make sure I knew what I thought I did.

Finally, publishing the questions makes it possible to answer the questions that the FAA has screwed up. The aeromedical section of the test is a total trainwreck, and having the questions told me what the FAA thought the correct answer was.

publishing the test bank questions was never a good idea....The idea of the knowlege test is that it is taken before the checkride, to help the student and flight instructor ascertain what has been learned throughout training, and what must be reviewed. Today that is not the case...many buy homestudy kits, prepart to take the knowlege test before they even begin flight training..the idea of a test is not a barrier one must overcome to get that certificate, its a tool...its an evaluation tool used by CFIs and ASIs/DPEs, if the student is serious about flight training, and not just doing the bare mins to pass, he will use it as an evaluation tool as well....the FAA should not post the test bank...period. All too often there is this idea that once your done with something...thats it. Checkride passed, insert books into closet, and your done....knowlege tests passed...same idea..only book that doesnt get put away is the FAR/AIM and the ASA oral test prep guide...there is a reason why when the FAA secretly changed the test questions everybody failed...it has nothing to do with tricky questions, and everything to do with not having the actual required knowlege and memorization of the test questions and answers. Gleim, ASA, Sheppard Air....they dont make any bones about how to use their software and hardware...it says explicitly this is used to get you to pass the knowlege test...in the opening statment

The resolution of this issue is obvious to anyone with specialized training in psychometrics and test development for certification and licensure. In the interest of public safety and legal defensibility, organizations such as the Institute for Credentialling Excellence (formerly the National Organization for Competency Assurance) have developed standards for development and administration of certification and licensure exams for professions, such as medicine and engineering, that help to assure that the exams and exam questions are valid, reliable, and legally defensible. It involves use of a psychometrician to guide subject matter experts in the development of the tests and test questions, and setting an appropriate passing point for each exam. Reliabilty is compromised when test questions are provided to candidates ahead of time--however, it is just fine to provide similar questions to the candidates ahead of time that cover all the knowledge areas to help the candidates learn and prepare for taking their exam. The performance of all test questions should be monitored by the psychometrician so that poorly-performing questions can be identified, pulled, or modified. New questions should be added on a trial basis and monitored for performance. Whoever is in charge of the FAA written exam program should know this and make sure it is done the right way. Dr. Tracy Tillman, Commerical, Multi, Inst., etc. Skymaster, Wilga, Maule, Blanik, Dragonfly, Alatus, MI26, 46MI airports.

The changing of how the written tests are managed is also a test of the industry. Some associations that will be involved in the review and input for test question development have a track record of doing what they can in the short term to boost membership so they can sell more advertising in their journals rather than what is best in the long run and the safety of its members. If they act responsibly this could greatly improve regulation if industry establishes a track record of integrity and is trusted and respected to provide relevant, practical, expert input and oversight. The question bank might as well be in the public domain. It doesn't take long for the braindumps on the web to post questions asked on any type of certification exam, especially where private, computerized testing centers are permitted to give the test. The only defense against this, to stop people from simply memorizing answers, to keep the test legitimate, is to have a question bank per test of at least 10,000 questions. We'll know that corrupt interests have dominated and put one over on the FAA if the number of questions asked per test remains about the same as it is now.

When I took the "written" eons ago, we didn't have access to the FAA's answers, at all, and there were few hawkers of test questions/answers. So we studied to learn the material, in hopes that by knowing the material, we'd pass the test. There were irrelevant questions then, too, a couple of questions about RMIs come to mind. The ADF/NDB questions were still very relevant then, and having shot a number of NDB approaches over the years, including relatively recently, I'd argue that they are still somewhat relevant, i.e., if the airplane is so equipped, the pilot should know how to use it.

But (for Brian) to argue that only GPS questions are relevant today shows a fundamental lack of knowledge about GPS. All GPS receivers will belly up simultaneously if the GPS signals are suddenly shut off simultaneously, which they can be in the name of national security. VORs and NDBs have to be shut off individually at the site. All non-military GPS receivers will be affected if Selective Availability is simultaneously turned on again, which substantially degrades GPS signals to non-military receivers. All GPS receivers are presently affected in a huge test area centered in New Mexico but stretching hundreds of miles in all directions, depending on altitude; this testing has been going on for the last couple of years across the southern tier of states. So if you haven't yet learned pilotage (chart reading) and ded reckoning (compass headings affected by wind direction and speed and elapsed time) and the airplane you fly has no other electronic navaids, I suggest that you stay in the pattern. To rely exclusively on only one kind of navigation method is both foolish and dangerous.

But (for Brian) to argue that only GPS questions are relevant today shows a fundamental lack of knowledge about GPS. All GPS receivers will belly up simultaneously if the GPS signals are suddenly shut off simultaneously, which they can be in the name of national security. VORs and NDBs have to be shut off individually at the site. All non-military GPS receivers will be affected if Selective Availability is simultaneously turned on again, which substantially degrades GPS signals to non-military receivers. All GPS receivers are presently affected in a huge test area centered in New Mexico but stretching hundreds of miles in all directions, depending on altitude; this testing has been going on for the last couple of years across the southern tier of states. So if you haven't yet learned pilotage (chart reading) and ded reckoning (compass headings affected by wind direction and speed and elapsed time) and the airplane you fly has no other electronic navaids, I suggest that you stay in the pattern. To rely exclusively on only one kind of navigation method is both foolish and dangerous.

Hey, Brian, with all these technological backups, you didn't get a chance to learn about North Korea blocking GPS signal for everyone recently (and there not the first ones to do it). It doesn't matter how many backups you have, if they all use the same source of data. Electricity supply is only one potential problem. That's what the old pilots have discovered a long time ago and are trying to warn less experience pilots about. Acquiring indepth comprehension of the tools pilot use is never done through learning the answers to questions and does increase safety. Nobody can manage risks they don't know risks exist. A good ground school might also explain the why's of the regulations concerning the use an iphone or a ipad as primary source of navigation... no can't do for very good reasons. On another note, understanding NDB navigation permits better use of the GPS data, which includes data similar to that of ADF because it essentially treats every waypoint as an NDB station and thus continuously provides bearing TO information. Finally, no pilot has been kill by too much knowledge. The reverse isn't true.

I'd never start a flight without my charts and E6-B. They don't require batteries and cannot be tampered with by any outside agency.

How do you plan a flight with a GPS only? How do you know what's to the sides of your courseline that may be of interest to you?

Linda

@Cary - With regard as to the GPS all failing suddenly (seriously?) or shut off simultaneously (much more "likely"), I think I covered that when I said "If all 3 go blank at the same time I'm landing the plane and finding out why I had 3 GPS failures inside of a 2 to 3 hour flight"

Pilotage and ded reckoning is part of every training program to the best of my knowledge - at least for the Sport and Private courses. Both of which are achievable on an iPad or iPhone. GPS on an iPad/iPhone is only one part of the features offered on these devices. A Foreflight (or WingsX) subscription provides the exact same paper maps in an electronic format. With that alone I can do the other two options. Additionally it provides all relevant AF/D information in a variety of different formats.

If we're going to play disaster scenarios that's fine - it's a good exercise.

@mireille - Gosh no I didn't hear about North Korea blocking GPS signals. I'll have to remember that the next time I'm flying my LSA in South Korea (the effected area). Again - GPS is only one of the "niceties" of devices such as an iPad. It does provide things other than GPS. In fact, you can buy an iPad without a GPS built-into it and still use it for navigation in a plane.

I'm not exactly sure what you're trying to make a point of with regard to your comment "A good ground school might also explain the why's of the regulations concerning the use an iphone or a ipad as primary source of navigation... no can't do for very good reasons." I think you're implying the "why's of why you shouldn't use the iPhone or iPad as a primary source of navigation. In which case this article (http://www.iflyamerica.org/safety_legally_use_ipad.asp) might be of interest.

My plane doesn't have the ability to use an NDB, or a VOR for that matter. So does that make my plane any less safe to fly? I think not considering there are tons of planes out there in the same, er - boat.

@ Linda - there's nothing wrong with how you're going about starting your flight w/ charts and an E6-B. I see the battery issue as a non-issue. Could it be an issue? Absolutely, but with proper resource management - and flying a plane is all about resource management I'll run out of fuel long before I will ever run out of battery juice.

Tampered by an outside agency? Seriously - such as?

If you're asking how to flight plan with GPS only, that tells me you haven't seen the tools that are available out there for doing exactly that. Find someone in your area that has an iPad with ForeFlight or WingsX and have them show you. If you have an AOPA account, go out to their website and look at their flight planning tools. Additionally - and to say it again, ForeFlight provides exactly the same map that you would have on paper in a digital format - both TAC and Sectionals. If you're comparing apples to apples, they are exactly the same thing - just one is on a dead tree - the other is burning electrons.

Two things: Tracy Tillman is correct if somewhat technical. Appropriately writing and validating questions is no job for amateurs. Most of the questions are written by subject matter experts but their writing and educational expertise aren't necessarily top of the line.

How about this for a compromise on publishing questions--a master statement file. The master statements would separate the chaff from the wheat of the body of knowledge required for each certificate. The FAA would be free to change questions at any time as long as the answer could be derived from the master statements. Hopefully, the the questions and their validity are vetted by professionals not just aviation subject matter experts.

Ground schools and oral and practical tests will still required to complete the pilot's knowledge base.

Publish the questions. The FAA technical writers configure questions sometimes like ODPs, that is to say, hard to understand. For many years I've had to interpret some of the questions, while scratching my head, and help the student understand. Publishing the questions, over 730, allows for a better instructional baseline where the FAA, the student and the flight instructor can work from. Who determines the questions and how are they put together? Understanding what the TECHNICAL WRITER configures gives way to understanding the core of the subject matter. Student and flight or ground instructors should not have to wonder what the FAA questions drift is.

It all comes down to how well the instructor has done his or her job to enable the student to function in the "real" world aviation circumstances.

What messes us up is that the real world is seldom as cut and dried as the test question world. Visibility and cloud ceilings for different airspace no problem on the test. It is one thing to read and learn what is marginal VFR. It is quite another to zipping around in it. This is where the instructor who is willing to show a student just how quickly things can turn ugly.

Forget the revelence of GPS vs ground based questions, how about weather questions on the test. Back when I took the private and instrument it seemed that the FAA exam was more about your ability to read a raw METAR and TAF vs how weather works and what it can do to you. Understanding weather is a life saving skill for a newly minted VFR pilot. They don't have the experience or the resources to draw on when bad things happen.

Brian, right on. The argument for paper charts and E6-B and against GPS is one that will go away in the near future as soon as the FAA does away with paper charts which I believe will happen sooner than later.

I wish to address my comments to three separate concerns:

Published vs. non-published Current vs. antiquated equipment / techniques Level of questions: Rote vs. HOTS

As a middle-aged, newly minted CFI who hopes to be a weekend part 61 instructor for the rest of her life, who is also a Visiting Fellow at a university, I care very much how the FAA will be designing its exams. I also hold degrees in both psychology and physics, and work in the electronic display industry, so am familiar with both educational theory, and the hard science that is part of the basis for flight and instrumentation.

Exams in the form of quizzes, such as found on the AOPA website, can be very useful teaching tools, as the process of information retrieval strongly reinforces knowledge already learned, and at the same time, provide powerful feedback and correction when knowledge is lacking or erroneously understood. However, the FAA exams are instituted as the first phase of the three phase process of evaluating the readiness for a certificate or rating: Written, Oral, Practical. The Written test then should adequately test if the candidate has the requisite book learning.

Thus, in a perfect world, the candidate would have read and understood all of the relevant reference material, aided by ground school or other additional training. They would NOT need to have published question banks. But, in this imperfect world, the capriciousness of the exam (read poorly written test questions) means that having good knowledge of the material, especially having also gained an understanding of the relative importance and developed the desired Higher Order Thinking Skills (HOTS), is not enough. Too many of the questions are of the type where the question developers have devised 'distractors' that look and in fact are, quite reasonable when using HOTS, but are designed to test one's rote learning of what is in reality, quite irrelevant. (e.g. What does it matter if one knows the exact words used to describe aeronautical decision making process, and in what order?)

On the other hand, it is very important that candidates know and understand, even if by only by rote, much of the regulatory material (e.g. airspeeds and visibility in each class airspace). My concern regarding what I have seen of the tests that I personally have taken recently is that many of the test questions seemed to have been devised by someone who is either not an aviator, or not a psychometrician. To properly design a valid test requires that the test questions be devised by someone who is both. The current tests are clearly outdated as to equipment and techniques (e.g. what commercial pilot today determines the distance to a VOR by the time it takes to see a change in bearing? We have DME and GPS for that today!)

Another problem I found with the test was frequency of subject areas. I personally took a test where five of the 100 questions were the same basic Weight&Balance problem with different numbers, one after the other. This over-weighted this subject area. The computer testing program should not be selecting randomly, but should be cognizant of the subject areas and properly weight each according to a pre-selected criteria.

In my opinion, we should NOT be publishing of the test bank broadly, but should be publishing them to a limited set of industry reviewers, comprised of aviation professionals, professional educators, and of course Gold Seal CFIs, such that each question may be reviewed and commented upon its currency and its usefulness as a screening question at the written level. Then, each question should continue to be tested for validity, etc. in the usual manner.

@Brian The test is "can you be trusted w/a airplane" - and not "can you purchase and master lots of toys"? Crying that the test is not relevant because it doesn't cater to your toy inventory is not an impressive argument. I am also a software engineer (yes, I code for Android and iOS). I can assure you from first hand experience that should you have an inflight issue then you will need to focus on flight and not on what the damn iPad says (assuming you even have time to ask the iPad). Honestly, in moderate chop I don't know how you even manage to interact w/those toys. Good luck w/your flying and try looking out the airplane some times. The scenery is quite nice.

"...Hopefully, the the questions and their validity are vetted by professionals not just aviation subject matter experts....

Nope. 630 in OKC writes the uestions and publishes them. I don't think they even have and instrutional systems desiner down there. The questions shout be vetted and validated by a non-involved third party.

Linda

In the late 60's and early 70's we used "Exam-O-Grams", published by the government to better understand specific issues. I found them to be a useful study tool since test questions were not available from the FAA. Providers of Aviation study materials gained their 'questions' database by conducting exit polls of students that recently completed a written exam. Educational providers requested you remember one test item that presented a problem and forward that info to them. The same technique could work in today's environment except that society is geared toward instant gratification. Schools could have a good sample of questions within a year, if the technique were used again.

@ Guy - Wow, I must've really hit a nerve with you. Your perspective on my "crying that the test is not relevant because it doesn't cater to your toy inventory" is a rather, shall I say "unique" and sadly misguided perspective. I'm sorry you feel that way, however wouldn't you want someone to be tested against information that is currently being implemented, or at least on the very fundamentals of it? You're in IT (as am I), I'm sure you'd be more than happy to hire someone with stellar scores in COBAL and FORTRAN for a Javascript position. I mean - after all, they are both programming right? So why would it matter that their knowledge of the coding isn't current with the times? Sure they could show up, fumble around and manage to make a go of it.... continuing....

....continuation...

Nowhere in any of my statements have I said that I spend all of my time head down fiddling with my iPad, or iPhone, or GPS (or paper map and E6B for that matter). These are merely tools to aid the flight process, and in reality they are typically tucked away during flight (the GPS is in the panel though). The thing that has stuck with me throughout my training and many FAA FAAST sessions has been "fly the plane first" - and that I do. The point I was attempting to make (in addition to making the test answers public) is the relevancy of the FAA tests have not kept up. To step away from the iPad discussion for a moment and give another example. In what world is the truncation of weather reports even necessary anymore? These codes were assembled back in the day when a single character occupied a very valuable space of memory and transmitting that information was a very difficult endeavor. Today I can find current weather charts through so many different vehicles it isn't even funny. Nearly all FBO's have a computer terminal hooked up to the Internet for accessing this information, complete with a full description - not just the truncated version.

Folks,

Let's not forget that a majority of the GA fleet still has steam guages.

Linda

Well the reality is that the FAA has stopped publishing most of their new questions which makes many publications obsolete. I have personally been teaching weekend classes for over 10 years and have put all of my knowledge in interactive programs that work just like a weekend class. Instead of having you focus on mindless/useless memorization of question and answer, we at Pilot Training Solutions give you all the knowledge required to pass your FAA knowledge tests in a simple format with time tested tricks that make the test a breeze and also help you save a ton of money on one-on-one ground school. Over the past 2 years we have helped more than a 1,000 students pass their tests and of those only 1 has failed the test after using our software. If you would like to check out our free demos and see if we fit your needs take a look at: http://passfaaexams.com/free-demo-downloads/ Lets see if we can help you with your aviation goals.

Lucas passfaaexams.com