PHPickerViewController Presentation

The first step in using PHPickerViewController is to create a configuration, an instance of PHPickerConfiguration. This is a struct, so in order to set any of its properties you’ll need a var declaration. If you’re going to want to access the user’s selected items directly from the photo library yourself as a PHAsset (as I’ll explain later in this chapter), you’ll need to initialize the configuration with a reference to the photo library:

var config = PHPickerConfiguration(photoLibrary: PHPhotoLibrary.shared())

But in general you won’t need to do that; my examples in this section don’t. Just initialize the configuration using init:

var config = PHPickerConfiguration()

The configuration has just two settable properties:

selectionLimit

How many items the user is permitted to select simultaneously. The default is 1. Specify a higher number to permit limited multiple selection, or 0 to mean unlimited multiple selection.

filter

What types of item the user will see in the picker view. Your choices (PHPickerFilter) are .images, .videos, and .livePhotos — or, to specify a combination of those, supply .any with an array of the permitted types. The default is no filter, meaning all three types will appear in the picker view.

In my tests, .livePhotos can be used positively but not negatively, because .images embraces live photos. In other words, if your filter is just .livePhotos, the user sees only live photos; but if your filter includes .images, the user will see live photos as well. (I regard this as a bug.)

Now create the picker, set its delegate, and present it:

var config = PHPickerConfiguration()
// ... set selectionLimit and filter as desired ...
let picker = PHPickerViewController(configuration: config)
picker.delegate = self
self.present(picker, animated: true)

PHPickerViewController Delegate

When the user has finished working with the PHPickerViewController, if the user actually selected one or more items and did not cancel, the delegate (PHPickerViewControllerDelegate) will receive this message:

• picker(_ :didFinishPicking:)

You must respond by dismissing the picker, which arrives in the first parameter. The second parameter is an array of PHPickerResult objects, one for each item the user selected; use these to learn of and retrieve the actual chosen images, videos, and live photos.

A PHPickerResult has just two properties:

assetIdentifier

A string suitable for obtaining from the photo library the PHAsset corresponding to the selected item. I’ll talk later about the photo library and PHAssets. In general you won’t need this, and it will be nil unless you initialized the PHPickerConfiguration with the photo library.

itemProvider

An NSItemProvider. This is the same class we talked about in connection with drag and drop in Chapter 10.

The idea behind the use of an NSItemProvider here is that no data has actually been handed to you; that’s a good thing, because photos can be big, and videos can be really big. Instead, the item provider is your gateway to obtain the data if you want it.

To learn what sort of data the item provider can provide, corresponding to what sort of item the user chose, look at the item provider’s registeredTypeIdentifiers, or just ask it whether it can provide data of the corresponding type (prov is the PHPickerResult’s itemProvider):

if prov.hasItemConformingToTypeIdentifier(UTType.movie.identifier) {
    // it's a video
} else if prov.canLoadObject(ofClass: PHLivePhoto.self) {
    // it's a live photo
} else if prov.canLoadObject(ofClass: UIImage.self) {
    // it's a photo
}

In that code, the order of the tests is deliberate. A live photo can be supplied in a simple UIImage representation, so if we test for images before live photos, we won’t learn that the result is a live photo.

Dealing with PHPickerViewController Results

In the PHPickerViewController delegate, you don’t have to extract the data for the user’s chosen items immediately. A PHPickerResult is a tiny lightweight object, so there is no penalty if you store it for later retrieval of the data (except that in the meantime the user might alter the photo library in a way that puts the item provider out of date).

But let’s assume that your goal is to extract the data immediately, and in particular that you want to display in your interface whatever the user chose. How would you do that?

The answer, obviously, depends on what sort of item this is. Suppose it’s a UIImage. Then we just ask the item provider to obtain the image and hand it to us. We could then, for example, proceed to display it in a UIImageView (again, prov is the PHPickerResult’s itemProvider):

prov.loadObject(ofClass: UIImage.self) { im, err in
    if let im = im as? UIImage {
        DispatchQueue.main.async {
            myImageView.image = im // or whatever
        }
    }
}

Observe that in that code I’ve stepped out to the main queue before doing anything to the interface. That’s crucial. loadObject(ofClass:) operates in the background, and calls its completion handler on a background thread; but we must talk to the interface on the main thread.

An interesting problem arises if you also want the metadata that accompanies the photo in the photo library. To get it, go back to the item provider and ask for the actual image data; then use the Image I/O framework to extract the metadata as a dictionary (import ImageIO, and see Chapter 23):

let im = UTType.image.identifier
prov.loadDataRepresentation(forTypeIdentifier: im) { data, err in
    if let data = data {
        let src = CGImageSourceCreateWithData(data as CFData, nil)!
        let d = CGImageSourceCopyPropertiesAtIndex(src,0,nil)
            as! [AnyHashable:Any]
        // ...
    }
}

The procedure for a live photo is exactly the same, except that if you want to display the live photo as a live photo, you need a PHLivePhoto:

prov.loadObject(ofClass: PHLivePhoto.self) { livePhoto, err in
    if let photo = livePhoto as? PHLivePhoto {
        DispatchQueue.main.async {
            // show the live photo
        }
    }
}

I’ll talk about how to display a live photo as a live photo later in this chapter.

Finally, what about a video? This is the hard case. In the first place, there is no class that represents a video, so we can’t call loadObject(ofClass:). And even if we could, we wouldn’t want to. The last thing we want on our hands is the entirety of a video’s data held in memory! What we want is the URL of a file on disk. That way, we can hand the URL to an AVPlayer or whatever mechanism we intend to use to display the video, as I described in Chapter 16.

Fortunately, an NSItemProvider will happily write the data directly to disk for us as it retrieves it, handing us the URL; all we have to do is call loadFileRepresentation(forTypeIdentifier:), which takes a completion handler with a URL parameter. However, the file in question is a temporary file that will be erased as soon as the completion handler ends — and we are going to need to step out to the main thread in order to display the video in our interface, so the completion handler will end while we are doing that. One solution would be to copy the file immediately to something that won’t be so temporary; but that would be time-consuming for a large video. A better approach is to step out to the main thread synchronously; that’s a rare thing to do, but it does no harm here, and it preserves the file long enough for our interface to take charge of it (and see Chapter 25 for more on this topic):

let movie = UTType.movie.identifier
prov.loadFileRepresentation(forTypeIdentifier: movie) { url, err in
    if let url = url {
        DispatchQueue.main.sync {
            // ... give the url to an AVPlayer or whatever ...
        }
    }
}

Observe that the call to loadFileRepresentation returns a Progress object that we could use to supply some additional interface if saving the video to disk is likely to take a long time.

A curious twist is that the video might be a live photo to which the Loop or Bounce effect has been applied. Implementing the looping, in that case, is up to us. We can detect this situation by examining the types registered with the item provider; the type I’m specifying is private, but I don’t know how else to learn that the video is supposed to loop:

let loopType = "com.apple.private.auto-loop-gif"
if prov.hasItemConformingToTypeIdentifier(loopType) { // it loops

The video can then be displayed from its URL using an AVPlayerLooper object (mentioned in Chapter 16). Start with an AVQueuePlayer rather than an AVPlayer, configure the AVPlayerLooper and retain it in an instance property, and use the AVQueuePlayer to show the video:

let av = AVPlayerViewController()
let player = AVQueuePlayer(url:url)
av.player = player
self.looper = AVPlayerLooper(
    player: player, templateItem: player.currentItem!)
// ... and so on ...

Querying the Photo Library

When you want to know what’s in the photo library, start with the photo entity class that represents the type of entity you want to know about. It will supply class methods whose names begin with fetch; you’ll pick the class method that expresses the kind of criteria you’re starting with. So to fetch one or more PHAssets, you’ll call a PHAsset fetch method; you can fetch by local identifier, by media type, or by containing asset collection. Similarly, to fetch one or more PHAssetCollections, you’ll call a PHAssetCollection fetch method; you can fetch by identifier, by type and subtype, or by whether they contain a given PHAsset.

In addition to the various fetch method parameters, you can supply a PHFetchOptions object letting you refine the results even further. You can set its predicate to limit your request results, and its sortDescriptors to determine the results order. Its fetchLimit can limit the number of results returned, and its includeAssetSourceTypes can specify where the results should come from, such as eliminating cloud items.

What you get back from a fetch method query is not images or videos but information. A fetch method returns a list of PHObjects of the type to which you sent the fetch method originally; these refer to entities in the photo library, rather than handing you an entire file (which would be huge and might take considerable time). The list itself is expressed as a PHFetchResult, which behaves very like an array: you can ask for its count, obtain the object at a given index, look for an object within the collection, extract objects into an array, and enumerate the objects with an enumerate method.

Let’s list all albums created by the user. An album is a PHAssetCollection, so the relevant class is PHAssetCollection:

let result = PHAssetCollection.fetchAssetCollections(
    with: .album, subtype: .albumRegular, options: nil)
let albums = result.objects(at: IndexSet(0..<result.count))
for album in albums {
    let count = album.estimatedAssetCount
    print("(album.localizedTitle!):",
        "approximately (count) photos")
}

In that code, we can learn how many assets are in each album only as its estimatedAssetCount. This is probably the right answer, but to obtain the real count, we’d have to dive one level deeper and fetch the album’s actual assets. Let’s do that: given an album, let’s list its contents. An album’s contents are its assets (photos and videos), so the relevant class is PHAsset:

let result = PHAsset.fetchAssets(in:album, options: nil)
let assets = result.objects(at: IndexSet(0..<result.count))
for asset in assets {
    print(asset.localIdentifier)
}

If the fetch method you need seems not to exist, don’t forget about PHFetchOptions. There is no PHAsset fetch method for fetching from a certain collection all assets of a certain type; you cannot specify, for instance, that you want all photos (but no videos) from the user’s Camera Roll. But you can perform such a fetch by setting a PHFetchOptions object’s predicate property. To illustrate, I’ll fetch ten ordinary photos (no videos, and no HDR photos) from the user’s Recent smart album:

let recentAlbums = PHAssetCollection.fetchAssetCollections(
    with: .smartAlbum, subtype: .smartAlbumRecentlyAdded, options: nil)
guard let rec = recentAlbums.firstObject else {return}
let options = PHFetchOptions()
let pred = NSPredicate(
    format: "mediaType == %d && !((mediaSubtype & %d) == %d)",
    PHAssetMediaType.image.rawValue,
    PHAssetMediaSubtype.photoHDR.rawValue,
    PHAssetMediaSubtype.photoHDR.rawValue)
options.predicate = pred // photos only, please, no HDRs
options.fetchLimit = 10 // let's not take all day about it
let photos = PHAsset.fetchAssets(in:rec, options: options)

Modifying the Library

Structural modifications to the photo library are performed through a change request class corresponding to the class of photo entity we wish to modify. The name of the change request class is the name of a photo entity class followed by “ChangeRequest.” For PHAsset, there’s the PHAssetChangeRequest class — and so on.

To use a change request, you’ll call a performChanges method on the shared photo library. Typically, that method will be performChanges(_:completionHandler:), which takes two functions. The first function, the changes function, is where you describe the changes you want performed; the second function, the completion function, is called back after the changes have been performed.

Each change request class comes with methods that ask for a change of some particular type. Here are some examples:

PHAssetChangeRequest

Class methods include deleteAssets(_:), creationRequestForAssetFromImage(atFileURL:), and so on.

If you’re creating an asset and what you’re starting with is raw data, call PHAssetCreationRequest.forAsset(). That gives you a PHAssetCreationRequest, a subclass of PHAssetChangeRequest that provides instance methods such as addResource(with:data:options:).

PHAssetCollectionChangeRequest

Class methods include deleteAssetCollections(_:) and creationRequestForAssetCollection(withTitle:).

In addition, there are initializers like init(for:), which takes an asset collection, along with instance methods addAssets(_:), removeAssets(_:), and so on.

A creationRequest class method also returns an instance of the change request class. You can throw this away if you don’t need it for anything. Its purpose is to let you perform further changes as part of the same batch. For example, once you have a PHAssetChangeRequest instance, you can use its properties to initialize the asset’s features, such as its creation date or its associated geographical location; those would be read-only if accessed through the PHAsset.

To illustrate, let’s create an album called “Test Album.” An album is a PHAssetCollection, so we start with the PHAssetCollectionChangeRequest class and call its creationRequestForAssetCollection(withTitle:) class method in the performChanges function. This method returns a PHAssetCollectionChangeRequest instance, but we don’t need that instance for anything, so we simply throw it away:

PHPhotoLibrary.shared().performChanges {
    let t = "TestAlbum"
    typealias Req = PHAssetCollectionChangeRequest
    Req.creationRequestForAssetCollection(withTitle:t)
}

(The class name PHAssetCollectionChangeRequest is very long, so I’ve shortened it with a type alias.)

It may appear, in that code, that we didn’t actually do anything — we asked for a creation request, but we didn’t tell it to do any creating. Nevertheless, that code is sufficient; generating the creation request for a new asset collection in the performChanges function constitutes an instruction to create an asset collection.

All the same, that code is rather silly. The album was created asynchronously, so to use it, we need a completion function (see Appendix C). Moreover, we’re left with no reference to the album we created. For that, we need a PHObjectPlaceholder. This minimal PHObject subclass has just one property — localIdentifier, which it inherits from PHObject. That’s sufficient to permit a reference to the created object to survive into the completion function, where we can do something useful with it, such as saving it off to an instance property:

var ph : PHObjectPlaceholder?
PHPhotoLibrary.shared().performChanges {
    let t = "TestAlbum"
    typealias Req = PHAssetCollectionChangeRequest
    let cr = Req.creationRequestForAssetCollection(withTitle:t)
    ph = cr.placeholderForCreatedAssetCollection
} completionHandler: { ok, err in
    if ok, let ph = ph {
        self.newAlbumId = ph.localIdentifier
    }
}

Now suppose we subsequently want to populate our newly created album. Let’s say we want to make the first asset in the user’s Recently Added smart album a member of our new album as well. No problem! First, we need a reference to the Recently Added album; then we need a reference to its first asset; and finally, we need a reference to our newly created album (whose identifier we’ve already captured as self.newAlbumId). Those are all basic fetch requests, which we can perform in succession, and we then use their results to form the change request:

// find Recently Added smart album
let result = PHAssetCollection.fetchAssetCollections(
    with: .smartAlbum, subtype: .smartAlbumRecentlyAdded, options: nil)
guard let rec = result.firstObject else { return }
// find its first asset
let result2 = PHAsset.fetchAssets(in:rec, options: nil)
guard let asset1 = result2.firstObject else { return }
// find our newly created album by its local id
let result3 = PHAssetCollection.fetchAssetCollections(
    withLocalIdentifiers: [self.newAlbumId], options: nil)
guard let alb2 = result3.firstObject else { return }
// ready to perform the change request
PHPhotoLibrary.shared().performChanges {
    typealias Req = PHAssetCollectionChangeRequest
    let cr = Req(for: alb2)
    cr?.addAssets([asset1] as NSArray)
}

A PHObjectPlaceholder has a further use. What if we created, say, an asset collection and wanted to add it to something (presumably to a PHCollectionList), all in one batch request? Requesting the creation of an asset collection gives us a PHAssetCollectionChangeRequest instance; you can’t add that to a collection. And the requested PHAssetCollection itself hasn’t been created yet! The solution is to obtain a PHObjectPlaceholder. Because it is a PHObject, it can be used as the argument of change request methods such as addChildCollections(_:).

Being Notified of Changes

When the library is modified, whether by your code or by some other means while your app is running, any information you’ve collected about the library — information which you may even be displaying in your interface at that very moment — may become out of date. To cope with this possibility, you should, perhaps very early in the life of your app, register a change observer (adopting the PHPhotoLibraryChangeObserver protocol) with the photo library:

PHPhotoLibrary.shared().register(self)

The outcome is that, whenever the library changes, the observer’s photoLibraryDidChange(_:) method is called, with a PHChange object encapsulating a description of the change. The observer can then probe the PHChange object by calling changeDetails(for:). The idea is that if you’re hanging on to information in an instance property, you can use what the PHChange object tells you to modify that information (and possibly your interface). The parameter can be one of two types:

A PHObject

The parameter is a single PHAsset, PHAssetCollection, or PHCollectionList you’re interested in. The result is a PHObjectChangeDetails object, with properties like objectBeforeChanges, objectAfterChanges, and objectWasDeleted.

A PHFetchResult

The result is a PHFetchResultChangeDetails object, with properties like fetchResultBeforeChanges, fetchResultAfterChanges, removedObjects, insertedObjects, and so on.

Suppose my interface is displaying a list of album names, which I obtained originally through a PHAssetCollection fetch request. And suppose that, at the time that I performed the fetch request, I also retained as an instance property (self.albums) the PHFetchResult that it returned. Then if my photoLibraryDidChange(_:) method is called, I can update the fetch result and change my interface accordingly:

func photoLibraryDidChange(_ changeInfo: PHChange) {
    if self.albums !== nil {
        let details = changeInfo.changeDetails(for:self.albums)
        if details !== nil {
            self.albums = details!.fetchResultAfterChanges
            // ... and adjust interface if needed ...
        }
    }
}

New in iOS 14, photoLibraryDidChange(_:) is called when your app has .limited access and the user changes the set of photos to which your app has access in the Select Photos interface. The reason is that, as far as your app is concerned, it is as if the user had altered the contents of the photo library.

Fetching Images

Sooner or later, you’ll probably want to go beyond information about the structure of the photo library and fetch an actual photo or video for display in your app. The process of obtaining an image can be time-consuming: not only may the image data be large, but also it may be stored in the cloud. Therefore you will typically supply a completion function that can be called back asynchronously with the data (see Appendix C).

To obtain an image, you’ll need an image manager, which you’ll get by calling the PHImageManager default class method. You then call a method whose name starts with request, supplying a completion function. For an image, you can ask for a UIImage or the original data; for a video, you can ask for an AVPlayerItem or an AVAsset configured for display of the video, or an AVAssetExportSession suitable for exporting the video to a new file (see Chapter 16). The result comes back to you as a parameter passed into your completion function.

func setUpInterface() {
    guard let asset = self.asset else { return }
    let opts = PHImageRequestOptions()
    opts.resizeMode = .exact
    PHImageManager.default().requestImage(for: asset,
        targetSize: CGSize(300,300), contentMode: .aspectFit,
        options: opts) { im, info in
            if let im = im {
                self.iv.image = im
            }
    }
}

let opts = PHLivePhotoRequestOptions()
opts.deliveryMode = .highQualityFormat
PHImageManager.default().requestLivePhoto(
    for: asset, targetSize: CGSize(300,300),
    contentMode: .aspectFit, options: opts) { photo, info in
        let v = PHLivePhotoView(frame: CGRect(20,20,300,300))
        v.contentMode = .scaleAspectFit
        v.livePhoto = photo
        self.view.addSubview(v)
}

func fetchMovie() {
    let opts = PHFetchOptions()
    opts.fetchLimit = 1
    let result = PHAsset.fetchAssets(with: .video, options: opts)
    guard let asset = result.firstObject else {return}
    PHImageManager.default().requestPlayerItem(
        forVideo: asset, options: nil) { item, info in
            if let item = item {
                DispatchQueue.main.async {
                    self.display(item:item)
                }
            }
    }
}
func display(item:AVPlayerItem) {
    let player = AVPlayer(playerItem: item)
    let vc = AVPlayerViewController()
    vc.player = player
    vc.view.frame = self.v.bounds
    self.addChild(vc)
    self.v.addSubview(vc.view)
    vc.didMove(toParent: self)
}

Editing Images

Astonishingly, PhotoKit allows you to change an image in the user’s photo library. Why is this even legal? There are two reasons:

• The user will have to give permission every time your app proposes to modify a photo in the library, and will be shown the proposed modification beforehand.

• Changes to library photos are undoable, because the original image remains in the database along with the changed image that the user sees, and the user can revert to that original at any time.

let options = PHContentEditingInputRequestOptions()
options.canHandleAdjustmentData = { adjustmentData in
    return adjustmentData.formatIdentifier == self.myidentifier
}
var id : PHContentEditingInputRequestID = 0
id = self.asset.requestContentEditingInput(with: options) { input, info in
    // ...
}

guard let input = input else {
    self.asset.cancelContentEditingInputRequest(id)
    return
}
self.input = input
let im = input.displaySizeImage! // show this to user during editing
if let adj = input.adjustmentData,
adj.formatIdentifier == self.myidentifier {
    if let vigNumber = try? NSKeyedUnarchiver.unarchivedObject(
        ofClass: NSNumber.self, from: adj.data),
    let vigAmount = vigNumber as? Double {
            // ... store vigAmount ...
    }
}
// ... present editing interface, passing it the vigAmount ...

let inurl = self.input.fullSizeImageURL!
let output = PHContentEditingOutput(contentEditingInput:self.input)
let outurl = output.renderedContentURL
var ci = CIImage(contentsOf: inurl, options: [.applyOrientationProperty:true])!
let space = ci.colorSpace!
if vignette >= 0.0 {
    let vig = MyVignetteFilter()
    vig.setValue(ci, forKey: "inputImage")
    vig.setValue(vignette, forKey: "inputPercentage")
    ci = vig.outputImage!
}

let data = try! NSKeyedArchiver.archivedData(
    withRootObject: vignette, requiringSecureCoding: true)
output.adjustmentData = PHAdjustmentData(
    formatIdentifier: self.myidentifier, formatVersion: "1.0", data: data)

try! CIContext().writeJPEGRepresentation(of:ci, to:outurl, colorSpace:space)

PHPhotoLibrary.shared().performChanges {
    typealias Req = PHAssetChangeRequest
    let req = Req(for: self.asset)
    req.contentEditingOutput = output // triggers alert
} completionHandler: { ok, err in
    if ok {
        // if we are displaying the image, redisplay it on the main thread
    } else {
        // user refused to allow modification, do nothing
    }
}

Photo Editing Extension

A photo editing extension is photo-modifying code supplied by your app that is effectively injected into the Photos app. When the user edits a photo from within the Photos app, your extension appears as an option and can modify the photo being edited.

To make a photo editing extension, create a new target in your app, specifying iOS → Application Extension → Photo Editing Extension. The template supplies a storyboard containing one scene, along with the code file for a corresponding UIViewController subclass. This file imports not only the Photos framework but also the Photos UI framework, which supplies the PHContentEditingController protocol, to which the view controller conforms. This protocol specifies the methods through which the runtime will communicate with your extension’s code.

A photo editing extension works almost exactly the same way as modifying photo library assets in general, as I described in the preceding section. The chief differences are:

• You don’t put a Done or a Cancel button into your editing interface. The Photos app will wrap your editing interface in its own interface, providing those buttons when it presents your view.

• You must situate the pieces of your code so as to respond to the calls that will come through the PHContentEditingController methods.

The PHContentEditingController methods are:

canHandle(_:)

You will not be instantiating PHContentEditingInput; the runtime will do it for you. Therefore, instead of configuring a PHContentEditingInputRequestOptions object and setting its canHandleAdjustmentData, you implement this method; you’ll receive the PHAdjustmentData and return a Bool.

startContentEditing(with:placeholderImage:)

The runtime has obtained the PHContentEditingInput object for you. Now it supplies that object to you, along with a very temporary initial version of the image to be displayed in your interface; you are expected to replace this with the PHContentEditingInput object’s displaySizeImage. Just as in the previous section’s code, you should retain the PHContentEditingInput object in a property, as you will need it again later.

cancelContentEditing

The user tapped Cancel. You may well have nothing to do here.

finishContentEditing(completionHandler:)

The user tapped Done. In your implementation, you get onto a background thread (the template configures this for you) and do exactly the same thing you would do if this were not a photo editing extension — get the PHContentEditingOutput object and set its adjustmentData; get the photo from the PHContentEditingInput object’s fullSizeImageURL, modify it, and save the modified image as a full-quality JPEG at the PHContentEditingOutput object’s renderedContentURL. When you’re done, don’t notify the PHPhotoLibrary; instead, call the completionHandler that arrived as a parameter, handing it the PHContentEditingOutput object.

During the time-consuming part of this method, the Photos app puts up a UIActivityIndicatorView, just as I suggested you might want to do in your own app. When you call the completionHandler, there is no alert asking the user to confirm the modification of the photo; the user is already in the Photos app and has explicitly asked to edit the photo, so no confirmation is needed.

Capture with UIImagePickerController

The simplest way to prompt the user to take a photo or video is with the UIImagePickerController class. It provides an interface that is effectively a limited subset of the Camera app. (In iOS 13 and before, UIImagePickerController was used also to let the user browse the photo library; now that iOS 14 provides the PHPickerViewController, the camera aspect of the UIImagePickerController will be the only aspect you’ll use.)

Here’s how to prepare:

1. Check isSourceTypeAvailable(_:) for .camera; if it is false, the user’s device has no camera or the camera is unavailable, so stop at this point.

2. Call availableMediaTypes(for:.camera) to learn whether the user can take a still photo (UTType.image.identifier), a video (UTType.movie.identifier), or both.

3. Instantiate UIImagePickerController, set its source type to .camera, and set its mediaTypes in accordance with which types you just learned are available. If your setting is an array of both types, the user will see a Camera-like interface allowing a choice of either one.

4. Set a delegate, adopting UINavigationControllerDelegate and UIImagePickerControllerDelegate.

Now you can present the picker:

let src = UIImagePickerController.SourceType.camera
guard UIImagePickerController.isSourceTypeAvailable(src)
    else {return}
guard = UIImagePickerController.availableMediaTypes(for:src) != nil
    else {return}
let picker = UIImagePickerController()
picker.sourceType = src
picker.mediaTypes = arr
picker.delegate = self
self.present(picker, animated: true)

For video, you can also specify the videoQuality and videoMaximumDuration. Moreover, these additional properties and class methods allow you to discover the camera capabilities:

isCameraDeviceAvailable:

Checks to see whether the front or rear camera is available, using one of these values as argument (UIImagePickerController.CameraDevice):

• .front

• .rear

cameraDevice

Lets you learn and set which camera is being used.

availableCaptureModes(for:)

Checks whether the given camera can capture still photos, video, or both. You specify the front or rear camera; returns an array of integers. Possible modes are (UIImagePickerController.CameraCaptureMode):

• .photo

• .video

cameraCaptureMode

Lets you learn and set the capture mode (still photo or video).

isFlashAvailable(for:)

Checks whether flash is available.

cameraFlashMode

Lets you learn and set the flash mode (or, for a movie, toggles the LED “torch”). Your choices are (UIImagePickerController.CameraFlashMode):

• .off

• .auto

• .on

When the view controller’s view appears, the user will see the interface for taking a picture, familiar from the Camera app, possibly including flash options, camera selection button, photo/video option (if your mediaTypes setting allows both), and Cancel and shutter buttons. If the user takes a picture, the presented view offers an opportunity to use the picture or to retake it. If you have set the picker’s allowsEditing to true, the user will have an opportunity to crop a still photo.

In the delegate, implement these two methods:

imagePickerControllerDidCancel(_:)

Dismiss the picker.

imagePickerController(_:didFinishPickingMediaWithInfo:)

Obtain what the user captured and dismiss the picker. The info: is a dictionary whose keys are of type UIImagePickerController.InfoKey. You’ll be interested in the following keys (any of which might be nil):

.mediaType

A string, either UTType.image.identifier or UTType.movie.identifier, telling you whether the user captured a still photo or a video.

.originalImage

A UIImage. If the user captured a still image, this is it. However, you should also check whether the .editedImage is not nil; if it is an actual UIImage, because you set allowsEditing to true and the user did in fact crop the image, you should use the .editedImage instead of the .originalImage.

.mediaMetadata

An NSDictionary containing the metadata for a captured image.

.mediaURL

If the user captured a video, the file URL where it has been saved.

The photo library was not involved in the process of media capture, so no user permission to access the photo library is needed. But if you now propose to save the media into the photo library, you will need permission. Suppose that the user takes a still image, and you now want to save it into the user’s Camera Roll album. Creating the PHAsset is sufficient:

func imagePickerController(_ picker: UIImagePickerController,
    didFinishPickingMediaWithInfo
    info: [UIImagePickerController.InfoKey : Any]) {
        var im = info[.originalImage] as? UIImage
        if let ed = info[.editedImage] as? UIImage {
            im = ed
        }
        let m = info[.mediaMetadata] as? NSDictionary
        self.dismiss(animated:true) {
            let mediatype = info[.mediaType]
            guard let type = mediatype as? String else {return}
            switch type {
            case UTType.image.identifier:
                if im != nil {
                    checkForPhotoLibraryAccess {
                        let lib = PHPhotoLibrary.shared()
                        lib.performChanges {
                            typealias Req = PHAssetChangeRequest
                            Req.creationRequestForAsset(from: im!)
                        }
                    }
                }
            default:break
            }
        }
}

In that code, the metadata associated with the photo is received (m), but nothing is done with it, and it is not folded into the PHAsset created from the image (im). To attach the metadata to the photo, use the Image I/O framework (import ImageIO) to make a copy of the image data along with the metadata. Now you can use a PHAssetCreationRequest to make the PHAsset from the data:

let jpeg = im!.jpegData(compressionQuality:1)
let src = CGImageSourceCreateWithData(jpeg as CFData, nil)!
let data = NSMutableData()
let uti = CGImageSourceGetType(src)!
let dest = CGImageDestinationCreateWithData(
    data as CFMutableData, uti, 1, nil)!
CGImageDestinationAddImageFromSource(dest, src, 0, m)
CGImageDestinationFinalize(dest)
let lib = PHPhotoLibrary.shared()
lib.performChanges {
    let req = PHAssetCreationRequest.forAsset()
    req.addResource(with: .photo, data: data as Data, options: nil)
}

You can customize the UIImagePickerController image capture interface. If you need to do that, you should probably consider dispensing entirely with UIImagePickerController and instead designing your own image capture interface from scratch, based around AV Foundation and AVCaptureSession, which I’ll introduce in the next section. Still, it may be that a modified UIImagePickerController is all you need.

In the image capture interface, you can hide the standard controls by setting showsCameraControls to false, replacing them with your own overlay view, which you supply as the value of the cameraOverlayView. That removes the shutter button, so you’re probably going to want to provide some new means of allowing the user to take a picture! You can do that through these methods:

• takePicture

• startVideoCapture

• stopVideoCapture

The UIImagePickerController is a UINavigationController, so if you need additional interface — possibly to let the user vet the captured picture before dismissing the picker — you can push it onto the navigation interface.

Capture with AV Foundation

Instead of using UIImagePickerController, you can control the camera directly using the AV Foundation framework (Chapter 16). You get no help with interface, but you get vastly more power than UIImagePickerController can give you. For stills, you can control focus and exposure directly and independently, and for video, you can determine the quality, size, and frame rate of the resulting movie.

To understand how AV Foundation classes are used for image capture, imagine how the Camera app works. When you are running the Camera app, you have, at all times, a “window on the world” — the screen is showing you what the camera sees. At some point, you might tap the button to take a still image or start taking a video; now what the camera sees goes into a file.

Think of all that as being controlled by an engine. This engine, the heart of all AV Foundation capture operations, is an AVCaptureSession object. It has inputs (such as a camera) and outputs (such as a file). It also has an associated layer in your interface. When you start the engine running, by calling startRunning, data flows from the input through the engine; that is how you get your “window on the world,” displaying on the screen what the camera sees.

As a rock-bottom example, let’s start by implementing just the “window on the world” part of the engine. Our AVCaptureSession is retained in an instance property (self.sess). We also need a special CALayer that will display what the camera is seeing — namely, an AVCaptureVideoPreviewLayer. This layer is not really an AVCaptureSession output; rather, the layer receives its imagery by association with the AVCaptureSession. Our capture session’s input is the default camera. We have no intention, as yet, of capturing anything to a file, so no output is needed:

self.sess = AVCaptureSession()
guard let cam = AVCaptureDevice.default(for: .video),
    let input = try? AVCaptureDeviceInput(device:cam)
    else {return}
self.sess.addInput(input)
let lay = AVCaptureVideoPreviewLayer(session:self.sess)
lay.frame = // ... some reasonable frame ...
self.view.layer.addSublayer(lay)
self.sess.startRunning()

Presto! Our interface now displays a “window on the world,” showing what the camera sees.

Suppose now that our intention is that, while the engine is running and the “window on the world” is showing, the user is to be allowed to tap a button that will capture a still photo. Now we do need an output for our AVCaptureSession. This will be an AVCapturePhotoOutput instance. We should also configure the session with a preset (AVCaptureSession.Preset) to match our intended use of it; in this case, the preset will be .photo.

So let’s modify the preceding code to give the session an output and a preset. We can do this directly before we start the session running. We can also do it while the session is already running (and in general, if you want to reconfigure a running session, doing so while it is running is far more efficient than stopping the session and starting it again), but then we must wrap our configuration changes in beginConfiguration and commitConfiguration. We should also run this sort of code on a dedicated background dispatch queue (see Chapter 25), but I’m not bothering to show that here:

self.sess.beginConfiguration()
guard self.sess.canSetSessionPreset(self.sess.sessionPreset)
    else {return}
self.sess.sessionPreset = .photo
let output = AVCapturePhotoOutput()
guard self.sess.canAddOutput(output)
    else {return}
self.sess.addOutput(output)
self.sess.commitConfiguration()

The session is now running and is ready to capture a photo. The user taps the button that asks to capture a photo, and we respond by telling the session’s photo output to capturePhoto(with:delegate:). To prepare for that, we’re going to need a reference to that output, which we can obtain from the session. Also, the first parameter is an AVCapturePhotoSettings object, which we should create and configure beforehand. So we start out like this:

guard let output = self.sess.outputs[0] as? AVCapturePhotoOutput
    else {return}
let settings = AVCapturePhotoSettings()

What sort of configuration might our AVCapturePhotoSettings need? Well, if we intend to display the user’s captured photo in our interface, we should request a preview image explicitly. It’s a lot more efficient for AV Foundation to create an uncompressed preview image of the correct size than for us to try to display or downsize a huge photo image. Here’s how we might ask for the preview image:

let pbpf = settings.availablePreviewPhotoPixelFormatTypes[0]
let len = // desired maximum dimension
settings.previewPhotoFormat = [
    kCVPixelBufferPixelFormatTypeKey as String : pbpf,
    kCVPixelBufferWidthKey as String : len,
    kCVPixelBufferHeightKey as String : len
]

Another good idea, when configuring the AVCapturePhotoSettings object, is to ask for a thumbnail image. This is different from the preview image: the preview image is for you to display in your interface, but the thumbnail image is stored with the photo and is suitable for rapid display by other applications. Here’s how to request a thumbnail image at a standard size (160×120):

settings.embeddedThumbnailPhotoFormat = [
    AVVideoCodecKey : AVVideoCodecType.jpeg
]

Just to make things more interesting, I’ll also specify explicitly that I want the camera to use automatic flash:

let supported = output.supportedFlashModes
if supported.contains(.auto) {
    settings.flashMode = .auto
}

When the AVCapturePhotoSettings object is fully configured, we’re ready to call capturePhoto(with:delegate:) at last:

output.capturePhoto(with: settings, delegate: self)

In that code, I specified self as the delegate (an AVCapturePhotoCaptureDelegate adopter). Functioning as the delegate, we will now receive a sequence of events. The exact sequence depends on what sort of capture we’re doing; in this case, it will be:

1. photoOutput(_:willBeginCaptureFor:)

2. photoOutput(_:willCapturePhotoFor:)

3. photoOutput(_:didCapturePhotoFor:)

4. photoOutput(_:didFinishProcessingPhoto:error:)

5. photoOutput(_:didFinishCaptureFor:)

The for: parameter throughout is an AVCaptureResolvedSettings object, embodying the settings actually used during the capture; for instance, we could use it to find out whether flash was actually used.

The delegate event of most interest to our example is the fourth one. This is where we receive the photo! It will arrive in the second parameter as an AVCapturePhoto object. This object contains a lot of information. It provides the resolved settings in its resolvedSettings property. Its previewPixelBuffer property contains the data for the preview image, if we requested one in our AVCapturePhotoSettings. We can extract the image data from the AVCapturePhoto by calling its fileDataRepresentation method. (There is also a longer form of the same method, fileDataRepresentation(with:), allowing you to do such things as modify the metadata and the thumbnail.)

In this example, we implement the fourth delegate method to store the preview image as a property, for subsequent display in our interface, and then save the actual image as a PHAsset in the user’s photo library:

func photoOutput(_ output: AVCapturePhotoOutput,
    didFinishProcessingPhoto photo:
    AVCapturePhoto, error: Error?) {
        if let cgim =
            photo.previewCGImageRepresentation()?.takeUnretainedValue() {
                let orient = // work out desired UIImage.Orientation
                self.previewImage = UIImage(
                    cgImage: cgim, scale: 1, orientation: orient)
        }
        if let data = photo.fileDataRepresentation() {
            let lib = PHPhotoLibrary.shared()
            lib.performChanges {
                let req = PHAssetCreationRequest.forAsset()
                req.addResource(with: .photo, data: data, options: nil)
            }
        }
    }

Image capture with AV Foundation is a huge subject, and our example of a simple photo capture has barely scratched the surface:

• AVCaptureVideoPreviewLayer provides methods for converting between layer coordinates and capture device coordinates; without such methods, this can be a very difficult problem to solve.

• You can scan bar codes, shoot video at 60 frames per second (on some devices), and more.

• You can turn on the LED “torch” by setting the back camera’s torchMode to AVCaptureTorchModeOn, even if no AVCaptureSession is running.

• You get direct hardware-level control over the camera focus, manual exposure, and white balance.

• You can capture bracketed images; starting in iOS 10, you can capture live images on some devices, and you can capture RAW images on some devices; and since iOS 11 even more new features have been introduced, such as depth-based image capture and multicamera capture.

There are very good WWDC videos about all this, stretching back over the past several years, and the AVCam-iOS and AVCamManual sample code examples are absolutely superb, demonstrating how to deal with tricky issues such as orientation that would otherwise be very difficult to figure out.