Apple has gone to great lengths to mirror the Mac OSX frameworks when designing iPhone OS. This makes the transition from the desktop to iPhone easier for Mac developers accustomed to naming conventions and runtime system of OSX. That said, there are several differences between the two that have serious consequences for portability. The most impacting is the differing CPU architecture; Macs are running on Intel or PowerPC CPUs, while iPhones are using ARM processors. Each CPU uses its own instruction set, requiring developers to compile their projects to the specific architecture they wish to target.
This chapter will focus on how to build Marsyas for the ARM processor, and other differences between Mac OS and iPhone OS that will impact Marsyas.
Implications and Scope of Marsyas on the iPhone
Before getting into the details of using Marsyas on the iPhone, one must determine if the reduced functionality of the framework will meet the needs of the application. Most notably, the iPhone implementation of CoreAudio is so different from the Mac version, it does not meet the requirements of Marsyas. This means Marsyas must be built without audio IO support. This may be a deal breaker for several types of applications. Further CoreMIDI does not yet exist on iPhone OS, so that is out too. These differences should imply that Marsyas is best used for analysis of audio, and actual MIDI or Audio IO should be implemented natively by the developer using CoreAudio and CoreMIDI. The example project in this chapter details this role.
Building a FAT libmarsyas
As previously mentioned, the most challenging aspect of using Marsyas on the iPhone is targeting the AR processor in the build process. Further complicating matters is the iPhone Simulator. Being a simulator (as opposed to an emulator), the iPhone simulator uses x86 (Intel) instructions. This means that testing an application in the simulator requires the project to be targeted at an x86 processor, while deploying to the actual device requires it to be targeted at the ARM processor. To achieve this, two versions of libmarsyas will be created: one with x86 instructions and one with ARM instructions. The two libraries will be merged into a single "FAT" library using the lipo tool. This FAT library is universal and can be included into projects targeted at both x86 and ARM processors. The drawback is that the library is twice as big as it needs to be for any single architecture.
The approach taken in this tutorial is to alter the Marsyas cmake settings in two iterations, first to build for ARM, then to build for x86. Finally the two libraries are combined into a single lib. Read the following subsections to understand this process or simply copy the following python script and execute it in the marsyas main directory.
#!/usr/bin/python
import os
import sys
# A script to set some CMake variables when building marsyas.
# We wish to target the iphone simulator (x86) / iphone device (arm)
# Parameters: Adjust these as required!
DEVROOT_PHONE = "/Developer/Platforms/iPhoneOS.platform/Developer"
DEVROOT_SIMULATOR = "/Developer/Platforms/iPhoneSimulator.platform/Developer"
## update this export with your most current version of the iPhone SDK
SDKROOT_PHONE = DEVROOT_PHONE + "/SDKs/iPhoneOS3.0.sdk"
SDKROOT_SIMULATOR = DEVROOT_SIMULATOR + "/SDKs/iPhoneSimulator3.0.sdk"
LIB_OUT_DIR = "fatLibraryOUTPUT"
## configures CMake to build for the arm iphone device
## and runs make to do so
def buildForIPhoneDevice(cleanAndRemake=False, withAudio=True):
print "---------------------------------------"
print "Building Marsyas for iPhone Device\n"
if(cleanAndRemake):
print "Removing old buildIphoneDevice build files"
os.system("rm -rf buildIphoneDevice")
## create a new build dir for this build and cd into it
os.mkdir("buildIphoneDevice")
os.chdir("buildIphoneDevice")
## build up the custom arguments for cmake:
cmake = "cmake "
# Compiler / linker etc
cmake += " -D CMAKE_CXX_COMPILER:FILEPATH="+ DEVROOT_PHONE + "/usr/bin/c++ "
cmake += " -D CMAKE_CXX_FLAGS:STRING=-arch\ armv6\ -pipe\ -no-cpp-precomp\ -isysroot\ "+ SDKROOT_PHONE + "\ -I" + SDKROOT_PHONE + "/usr/lib/gcc/arm-apple-darwin9/4.2.1/include/\ -I"+ SDKROOT_PHONE + "/usr/include/\ "
cmake += " -D CMAKE_AR:FILEPATH=/usr/bin/ar "
cmake += " -D CMAKE_C_COMPILER:FILEPATH="+ DEVROOT_PHONE + "/usr/bin/gcc "
cmake += " -D CMAKE_C_FLAGS:STRING=-arch\ armv6\ -pipe\ -no-cpp-precomp\ -isysroot\ "+ SDKROOT_PHONE + "\ -I" + SDKROOT_PHONE + "\ -I" + SDKROOT_PHONE + "/usr/lib/gcc/arm-apple-darwin9/4.2.1/include/\ -I"+ SDKROOT_PHONE + "/usr/include/ "
cmake += " -D CMAKE_EXE_LINKER_FLAGS:STRING=--host=arm-apple-darwin "
cmake += " -D CMAKE_MODULE_LINKER_FLAGS:STRING= "
cmake += " -D CMAKE_LINKER:FILEPATH="+ DEVROOT_PHONE + "/usr/bin/ld "
cmake += " -D CMAKE_OSX_ARCHITECTURES:STRING=armv6 " ## from i386
#isysroot used for universal binary support
cmake += " -D CMAKE_OSX_SYSROOT:PATH=" + DEVROOT_PHONE + "/SDKs/iPhoneOS3.0.sdk " # from /Developer/SDKs/MacOSX10.5.sdk
# Differences exist between Mac and iPhone CoreAudio.frameworks. be warned.
cmake += " -D COREAUDIO_LIBRARY:FILEPATH=" + SDKROOT_PHONE + "/System/Library/Frameworks/CoreAudio.framework "
if withAudio:
cmake += " -D MARSYAS_AUDIOIO:BOOL=ON "
else:
cmake += " -D MARSYAS_AUDIOIO:BOOL=OFF "
cmake += " -D COREFOUNDATION_LIBRARY:FILEPATH=" + SDKROOT_PHONE + "/System/Library/Frameworks/CoreFoundation.framework "
# Apple has not implemented Core MIDI on iphone yet:
# cmake += " -D COREMIDI_LIBRARY:FILEPATH=" + SDKROOT_PHONE + "/System/Library/Frameworks/CoreMidi.framework "
cmake += " -D MARSYAS_MIDIIO:BOOL=OFF " # enable to ON when implemented
cmake += " -D MARSYAS_BUILD_APPS:BOOL=OFF "
# Open GL for iPhone is questionable, it only supports a subset of options...
# replace OpenGL.framework with OpenGLES.framework. AGL.framework not implemented on iPHone...
cmake += " -D OPENGL_INCLUDE_DIR:PATH=" + SDKROOT_PHONE + "/System/Library/Frameworks/OpenGLES.framework "
cmake += " -D OPENGL_gl_LIBRARY:FILEPATH=" + SDKROOT_PHONE + "/System/Library/Frameworks/OpenGLES.framework "
# cmake += " -D OPENGL_glu_LIBRARY:FILEPATH=" + SDKROOT_PHONE + "/System/Library/Frameworks/AGL.framework " enable when ready
cmake += " -D WITH_OPENGL:BOOL=OFF " # endable when ready
# MP3 support needs the libmad library compiled for arm on the iphone device:
cmake += " -D WITH_MAD:BOOL=OFF "
cmake += " -D WITH_PNG:BOOL=OFF "
cmake += " -D WITH_QT:BOOL=OFF "
cmake += " -D WITH_SWIG:BOOL=OFF "
cmake += " -D WITH_VAMP:BOOL=OFF "
cmake += " -D WITH_VORBIS:BOOL=OFF "
cmake += " ../src"
## execute cmake to generate the makefiles:
print "generating makefiles with cmake"
os.system(cmake)
## run make to build marsyas
print "Making marsyas"
os.system("make marsyas")
## move the generated lib file to a safe place
os.system("mv lib/libmarsyas.a ../" + LIB_OUT_DIR + "/libmarsyas.a.arm")
## go back to the original working directory:
os.chdir("..")
## configures CMake to build for the x86 iphone simulator
## and runs make to do so
def buildForIPhoneSimulator(cleanAndRemake=False, withAudio=True):
print "---------------------------------------"
print "Building Marsyas for iPhone Simulator\n"
if(cleanAndRemake):
print "Removing old buildIphoneSimulator build files"
os.system("rm -rf buildIphoneSimulator")
## create a new build dir for this build and cd into it
os.mkdir("buildIphoneSimulator")
os.chdir("buildIphoneSimulator")
## build up the custom arguments for cmake:
cmake = "cmake "
# Compiler / linker etc
cmake += " -D CMAKE_CXX_COMPILER:FILEPATH=/usr/bin/c++ "
cmake += " -D CMAKE_CXX_FLAGS:STRING= "
cmake += " -D CMAKE_AR:FILEPATH=/usr/bin/ar "
cmake += " -D CMAKE_C_FLAGS:STRING= "
cmake += " -D CMAKE_C_COMPILER:FILEPATH=/usr/bin/gcc "
cmake += " -D CMAKE_EXE_LINKER_FLAGS:STRING= "
cmake += " -D CMAKE_MODULE_LINKER_FLAGS:STRING= "
cmake += " -D CMAKE_LINKER:FILEPATH=/usr/bin/ld "
cmake += " -D CMAKE_OSX_ARCHITECTURES:STRING=i386 "
#isysroot used for universal binary support
cmake += " -D CMAKE_OSX_SYSROOT:PATH=/Developer/SDKs/MacOSX10.5.sdk "
# Differences exist between Mac and iPhone CoreAudio.frameworks. be warned.
cmake += " -D COREAUDIO_LIBRARY:FILEPATH=" + SDKROOT_SIMULATOR + "/System/Library/Frameworks/CoreAudio.framework "
if withAudio:
cmake += " -D MARSYAS_AUDIOIO:BOOL=ON "
else:
cmake += " -D MARSYAS_AUDIOIO:BOOL=OFF "
cmake += " -D COREFOUNDATION_LIBRARY:FILEPATH=" + SDKROOT_SIMULATOR + "/System/Library/Frameworks/CoreFoundation.framework "
# Apple has not implemented Core MIDI on iphone yet:
# cmake += " -D COREMIDI_LIBRARY:FILEPATH=" + SDKROOT_SIMULATOR + "/System/Library/Frameworks/CoreMidi.framework "
cmake += " -D MARSYAS_MIDIIO:BOOL=OFF " # enable to ON when implemented
cmake += " -D MARSYAS_BUILD_APPS:BOOL=OFF "
# Open GL for iPhone is questionable, it only supports a subset of options...
# replace OpenGL.framework with OpenGLES.framework. AGL.framework not implemented on iPHone...
cmake += " -D OPENGL_INCLUDE_DIR:PATH=" + SDKROOT_SIMULATOR + "/System/Library/Frameworks/OpenGLES.framework "
cmake += " -D OPENGL_gl_LIBRARY:FILEPATH=" + SDKROOT_SIMULATOR + "/System/Library/Frameworks/OpenGLES.framework "
# cmake += " -D OPENGL_glu_LIBRARY:FILEPATH=" + SDKROOT_SIMULATOR + "/System/Library/Frameworks/AGL.framework " enable when ready
cmake += " -D WITH_OPENGL:BOOL=OFF " # endable when ready
# MP3 support needs the libmad library compiled for arm on the iphone device:
cmake += " -D WITH_MAD:BOOL=OFF "
cmake += " -D WITH_PNG:BOOL=OFF "
cmake += " -D WITH_QT:BOOL=OFF "
cmake += " -D WITH_SWIG:BOOL=OFF "
cmake += " -D WITH_VAMP:BOOL=OFF "
cmake += " -D WITH_VORBIS:BOOL=OFF "
cmake += " ../src"
## execute cmake to generate the makefiles:
print "generating makefiles with cmake"
os.system(cmake)
## run make to build marsyas
print "Making marsyas"
os.system("make marsyas")
## move the generated lib file to a safe place
os.system("mv lib/libmarsyas.a ../" + LIB_OUT_DIR + "/libmarsyas.a.i386")
## go back to the original working directory:
os.chdir("..")
## creates one fat static library with the contents of the arm
## and x86 versions of libmarsyas.a
def createFatStaticLibrary():
print "Creating the 'fat' marsyas library for i386 and arm "
os.system(DEVROOT_PHONE + "/usr/bin/lipo -arch arm " + LIB_OUT_DIR + "/libmarsyas.a.arm -arch i386 " + LIB_OUT_DIR + "/libmarsyas.a.i386 -create -output " + LIB_OUT_DIR + "/libmarsyas.a")
## main body:
if __name__ == '__main__':
os.system("mkdir " + LIB_OUT_DIR)
buildForIPhoneDevice(cleanAndRemake=True, withAudio=False)
buildForIPhoneSimulator(cleanAndRemake=True, withAudio=True)
createFatStaticLibrary()
Scripting Multiple cmake builds
The script defines three functions:
if __name__ == '__main__':
os.system("mkdir " + LIB_OUT_DIR)
buildForIPhoneDevice(cleanAndRemake=True, withAudio=False)
buildForIPhoneSimulator(cleanAndRemake=True, withAudio=True)
createFatStaticLibrary()
The script creates a new directory to hold the three versions of libmarsyas (ARM, x86 and combined). Each function creates the respective version of the library. At this point, withAudio may as well be set to false since Marsyas is not able to use the iPhone version of CoreAudio (runtime errors will result if it is attempted).
DEVROOT_PHONE = "/Developer/Platforms/iPhoneOS.platform/Developer"
DEVROOT_SIMULATOR = "/Developer/Platforms/iPhoneSimulator.platform/Developer"
## update this export with your most current version of the iPhone SDK
SDKROOT_PHONE = DEVROOT_PHONE + "/SDKs/iPhoneOS3.0.sdk"
SDKROOT_SIMULATOR = DEVROOT_SIMULATOR + "/SDKs/iPhoneSimulator3.0.sdk"
Four constants are also set. These help the script find the tools needed by cmake / make to build Marsyas. These constants should be altered to reflect the current system.
The buildForIphoneSimulator functions is fairly straightforward. First several cmake variables are set that reflect the reduced functionality of the iPhone SDK. Read through the parameters to get a sense of what is available and what is not (e.g no openGL, CoreAudio, CoreMIDI, mp3 support). Next cmake <varlist> ../src is called to create the CMakeCache.txt file and various build scripts for the simulator. Finally make marsyas is called to build the Marsyas library.
The buildForIphoneDevice function is more interesting, but works pretty much the same as the previous function. The only differences are the changes to the compiler and linker flags. The script instructs cmake to use the iPhone gcc compiler and to set the target architecture to armv6. Again reading through the set variables, and referencing the gcc / cmake documentation should make its intent clear.
The script concludes by calling lipo to create a FAT static library containing both instruction sets. This final library can be found in the marsyas/FATlibraryOUTPUT directory.
Create an iPhone Project
The example project will be another selection from the Marsyas documentation that shows how to extract data from the MarSystem network. Notice how the role of Marsyas has changed here. It is being used for audio analysis, not so much for actual playback. Marsyas should be relied on for these types of features, while any actual audio playback / recording should be separately implemented using CoreAudio and AudioToolbox frameworks.
Begin by creating a new iPhone project template. Select the View Based Application:
Take the usual steps to incorporate Marsyas into a project:
- Link against CoreFoundation (CoreAudio does not work properly, and CoreMIDI does not exist).
- Link against the FAT libmarsyas.a created to target both x86 and ARM.
- Update the header search path to include the Marsyas header files.
Implement the ViewController
The project template will have created a default ViewController: <projectName>ViewController.m. Rename this file to <projectName>ViewController.mm to let the compiler know that there will be C++ code as well as Objective-C code. Update the header and implementation files with the following code:
// MarsyasIphoneViewController.h
#import
@interface MarsyasIphoneViewController : UIViewController {
IBOutlet UILabel* filenameLabel;
NSString* filename;
}
@property(nonatomic, retain) IBOutlet UILabel* filenameLabel;
@property(nonatomic, retain) NSString* filename;
-(IBAction)printData:(id)sender;
@end
// MarsyasIphoneViewController.mm
#import "MarsyasIphoneViewController.h"
#import
#include "MarSystemManager.h"
using namespace Marsyas;
using namespace std;
@implementation MarsyasIphoneViewController
@synthesize filename;
@synthesize filenameLabel;
#pragma mark lifecycle methods
-(void)viewDidLoad{
std::string aString = "Hello World";
std::cout << aString << std::endl;
self.filename = [[NSBundle mainBundle] pathForResource:@"song" ofType:@"wav"];
[filenameLabel setText:filename];
}
- (void)dealloc {
self.filename = nil;
self.filenameLabel = nil;
[super dealloc];
}
#pragma mark actions
-(IBAction)printData:(id)sender{
string sfName = [filename UTF8String];
MarSystemManager mng;
MarSystem* pnet = mng.create("Series", "pnet");
// standard network
pnet->addMarSystem(mng.create("SoundFileSource", "src"));
pnet->updctrl("SoundFileSource/src/mrs_string/filename", sfName);
pnet->addMarSystem(mng.create("Spectrum","spk"));
pnet->addMarSystem(mng.create("Gain", "g2"));
while ( pnet->getctrl("SoundFileSource/src/mrs_bool/notEmpty")->to() )
{
pnet->tick();
// gets data from the Spectrum for read only!
const realvec& processedData =
pnet->getctrl("Spectrum/spk/mrs_realvec/processedData")->to();
cout << "Original Spectrum = " << processedData << endl;
// if we need to get the Spectrum and modify it, here is the way
// to do it. Notice that we must open a new scope using curly
// brackets so that MarControlAccessor is automatically destroyed
// when we are finished modifing the realvec control.
{
MarControlAccessor
acc(pnet->getctrl("Spectrum/spk/mrs_realvec/processedData"));
realvec&
processedData2 = acc.to();
// we can now write to processedData without worries of
// breaking encapsulation
processedData2 *= 2.0;
cout << "Modified Spectrum = " << processedData2 << endl;
}
}
delete pnet;
}
@end
Wire the NIB file in Interface Builder
Open the corresponding <projectName>ViewController.xib file in Interface Builder. Create an interface similar to the following:
Wire the UILabel to the File Owner's filenameLabel outlet, and wire the UIButton to the printData action. Let the label take up a large space and set it to show several lines of text (at least 5).
Add an Audio Resource
Find some small wav file from the internet and drag it into the project's resources folder. Select the copy option and set the reference type to relative to project. Give the wav file the name song.wav.
Build and Test the Program
Finally, click the Build and Go hammer. The iPhone simulator should launch and present the UI that was created. Clicking on button should print out the MarSystem output to the console. While this example is somewhat impractical, it shows how to build for the iPhone and how to make use of the data generated by Marsyas. It also shows how to work the Marsyas C++ code into an Objective-C environment.
]]>
